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TITLE: TRANSCRIPTION SYSTEM FOR 
MANIPULATING REAL-TIME TESTIMONY 

BACKGROUND OF THE INVENTION 

This invention relates to a transcription system used by- 
court reporters; and, more particularly, it relates to a method 
and apparatus incorporating an automatic transcription system for 
providing real-time use and manipulation of transcribed testimony 
.5 by attorneys, judges, court reporters, witnesses and clients. 

As is well known, legal proceedings such as a deposition or 
trial involve the participation of, among others, an examining 
attorney who asks questions and a witness who must answer 
("testify") while under oath. These answers ("testimony") are 
10 recorded . by the court reporter, along with the associated 
questions and related conversation, using a stenographic 
recorder. A stenographic recorder is a machine which provides 
a set of keys which are stroked by the court reporter in various 
combinations and sequences to represent a spoken word. To 
15 provide a backup to the key- strokes, court reporters use a tape 
recorder to record the entire proceeding. " 

Older versions of the stenographic recorder only record the 
court reporter's key-strokes much the way a computer printer 
does, creating a paper tape containing the printed key-strokes 
20 as a record of the legal proceeding. To the untrained eye, the 
printed key-strokes are incomprehensible. After the legal 
proceeding ends, the court reporter reads the paper tape and 
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manually transcribes each printed key- stroke back into the words 
that were spoken, creating a readable transcript of the 
testimony. This manual process is herein called "manual, post- 
processed transcription". 
5 Newer versions of stenographic recorders have been 

developed and . linked to computer aided transcription ("CAT") 
systems to help automate the manual transcription process. 
Instead of solely using paper tape recording methods, the 
stenographic recorder also electronically stores key- strokes in 

10 built-in memory or on disk. After using such a newer recorder, 
the court reporter returns to his office and transfers the 
electronically stored key-strokes to his CAT system for 
transcription. Although the bulk of electronically stored key- 
strokes may be translated automatically, the court reporter must 

15 still work interactively with the CAT system to translate those 
key-strokes which the CAT system could not recognize. This 
nearly automated transcription process is herein called 
"automatic, post-processed transcription". 

The most recent versions of the stenographic recorder also 

20 accommodate "automatic, parallel transcription", by 
electronically sending each electronically registered key- stroke 
via a telephone line to a remote CAT system during the legal 
proceeding.. This system requires two court reporters working in 
parallel: one using the stenographic recorder to record the 

25 testimony; and the other, who is familiar with the first 
reporter's key-stroking style, interacting with the CAT system 
to carry out the transcription of each key-stroke received. 
Working in parallel, the court reporters might produce a readable 
transcript within a few hours after the proceeding has ended. 

30 For all of the transcription systems mentioned above, court 

reporters often use the tape recorder to aid in translating 

SUBSTITUTE SHEET (RULE 26) Ei005957 



WO 94/22246 PCT/US94/03218 

-3- 

misheard words and . untranslated key-strokes. To locate the 
corresponding audio on a recorded tape often wastes a great deal 
of the court reporter's time. 

Automatic, parallel transcription systems have also been 
5 unsuccessfully used in a configuration to assist a deaf witness 
in legal proceedings. Taking the place of the second, parallel 
court reporter in the aforementioned normal scenario, the deaf 
witness is placed in front of the CAT system. The CAT system 
translates and displays the bulk of the key-strokes and sequences 
10 in real-time so that the deaf person can read the questions 
asked. One major problem found in this configuration is that the 
untranslated key-strokes are displayed in an undecipherable form 
to all but a court reporter. A relatively large number of the 
questions asked become unintelligible. Furthermore, the fact 
15 that the witness is deaf complicates this matter to a point of 
failure. For every unintelligible question encountered, the 
court reporter must stop using the stenographic recorder, read 
the computer screen and then write down the translation by hand 
on a piece of paper. for the witness. As an additional problem 
20 the witness's own words annoyingly echo back to the computer 
screen within a short delay after they are spoken, making it very 
difficult for the witness to testify without distraction. 

The development of the automatic, post-processed and 
parallel transcription system was driven solely by the court 
25 reporter's need to accelerate the transcription process. Using 
these systems, the court reporter operates more efficiently, 
handles a larger volume of work and, as a result, reaps greater 
profit. Using any of these transcription systems, however, a 
readable transcript is never received by an attorney, judge or 
30 witness until some time after the immediate legal proceeding has 
ended. 



SUBSTITUTE SHEET (MIIE26) 



WO 94/22246 PCT/US94/03218 

-4- 

There are a variety of problems facing attorneys, judges 
and witnesses relating to the development of testimony taken in 
a legal proceeding. The focal point of most of these problems 
involves the lack of a real-time, readable record of a witness's 
5 testimony while that witness is undergoing examination. Reliance 
must be based upon either the minimal assistance the court 
reporter might offer or notes taken which often prove to be 
burdensome, confusing and untrustworthy. Further details of 
these problems become apparent when evaluating the type of legal 
10 proceeding involved. 

A deposition proceeding often involves the participation 
of a witness, court reporter, examining attorney, associate 
examining attorney, defending attorney and associate defending 
attorney. Generally, the examining attorney asks questions to 

.15 which the witness responds with answers. The associate examining 
attorney assists the examining attorney in conducting this 
inquiry. The defending attorney, with assistance from the 
associate defending attorney, evaluates the questions being asked 
and raises objections when legally improper questions are 

.20 detected. If a legally improper question occurs but is not 
detected, the corresponding answer may be used to the detriment 
of the defending attorney's case. During the entire proceeding, 
the court reporter records all of the questions, answers, 
objections and discussions held during the deposition. 
25 In parallel with the court reporter's recording of the 

proceeding, all attorneys rapidly take notes in which they 
attempt to summarize what they believe to be the important 
factors of what they believe they heard the witness say. 
Although these notes provide a necessary, working record to aid 
30 . in conducting the deposition, they are often incomplete, 
mischaracterize or, even worse, completely misconstrue a 
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witness's answers... Relying on such notes often proves to be 
detrimental. 

If an attorney does not understand an answer, he often asks 
the court reporter to read the answer back from the paper tape 
5 record.. The examining or defending attorney may also ask for the 
last answer to be read back if the answer was so lengthy that 
notes were not taken fast enough to keep up with the witness. 
Similarly, either .attorney may ask the court reporter to read 
back the last, question for a confused witness. When so 
10 instructed, the court reporter stops recording, picks up the 
paper tape output from the stenographic recorder, searches for 
the portion of the record at issue, and reads the stenographic 
key- strokes. This reading is often broken with misplaced delays 
in pronunciation between phonemes and words making it difficult 
15 to understand. Furthermore, because searching for a portion of 

........ ,\ the record in a series of printed key-strokes proves to be very 

difficult and time consuming, attorneys are forced to limit 
requests to have only the most recent Q & A's read back. 

More specifically, the examining attorney takes notes in 

20 order to help formulate questions during a deposition. These 
notes are referenced particularly as the witness reveals 
additional evidence about the same subject. If the deposition 
runs for several days, the examining attorney often reviews each 
day's notes during the evening in his hotel room in preparation 

.25 for the next day of deposition. Reviewing cryptic notes taken 
many hours earlier often causes further misunderstandings which 
may not be detected until a readable transcript is received. 
Operating from unreliable notes, the examining attorney might 
incorrectly: 1) proceed with irrelevant lines of questioning; 2) 

30 reopen resolved, important issues -- tipping-off the defending 
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attorneys; or 3) abandon unresolved, important lines of 
questioning. 

A major goal of the examining attorney is to establish the 
proper form of Q & A's which relate to. important issues in a 
5 case. In furthering this goal, the examining attorney often 
attempts to embody one or more of the witness's prior answers as 
part of a new, summarizing question. Because this new question 
is often based upon unreliable notes or poor memory, the new 
question mischaracterizes the prior answers. In response, the 
10 defending attorney may correctly object if he detects the 
mischaracterization from his potentially unreliable notes and 
poor memory. The objection forces the examining attorney to 
consider either asking the court reporter to try to find and read 
back the actual Q & A's at issue or re-asking the entire line of 
. 15 questioning again. . More importantly, this objection warns the 
. witness to carefully scrutinize each question relating to this 
subject before answering. As. a result, a battle usually results 
over superfluous terms and phrases, and obtaining proper form is 
prevented. Furthermore, even if the examining attorney believes 
20 proper form has been achieved, he is often reluctant to verify 
his belief by having the Q & A read back because it serves to 
alert the defending. attorneys that important information may have 
come to light. Thus, improper form on critical issues often 
results. 

25 Leeching away his time for, and therefore the reliability 

of, his note taking-, the examining attorney is also faced with 
a multitude of other tasks such as: 1) analyzing the response of 
the witness; 2) formulating a follow up question in view of the 
witnesses response; 3.) analyzing the defending attorneys 

30 objections to the pending question; 4) evaluating the demeanor 
of the witness upon each Q & A; 5) reviewing an outline for the 
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next preplanned question or line of questioning; and 6) writing 
further notes regarding future lines of questioning to be asked 
or reviewed that evening at the hotel. 

To relieve him of some of the burden, the examining 
5 attorney often brings an associate examining attorney with him. 
The associate does several things including: 1) taking notes to 
fill gaps in the examining attorneys notes at a break or later 
that evening at the hotel when the two attorneys plan the next 
day's questions; 2) ordering and finding documentary exhibits for 
10 the examining attorney; 3) writing messages on small pieces of 
paper (usually POST- IT brand notes) and passing them to the 
examining attorney; 4) monitoring the passed messages so as to 
withdraw, supplement . or replace them when necessary; and 5) 
"keeping track of the documentary exhibits entered into evidence. 
15 While relieving some of the burden upon the examining 

attorney, the associate examining attorney creates additional 
burdens . For example, in suggesting a follow-up question, the 
associate must rapidly compose a brief message and immediately 
obtain the attention of the examining attorney before the 
20 examining attorney changes the line of questioning. As a result, 
the- examining attorney sometimes forgets his own line of 
questioning. Moreover, these messages tend to be illegible and 
too brief for the examining attorney to understand. Without 
supplementation, these messages serve no purpose but to distract. 
25 To suggest new lines of questioning, the associate may 

casually draft a rather complete message intending for the 
examining attorney to read the message at the examining 
attorney's leisure. However, the mere act of passing the note 
tends to distract the attention of the examining attorney. 
30 The associate attorney may also pass messages to help 

counter the defending attorneys' objections or pointing out the 
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witness's demeanor in response to a question when the examining 
attorney is looking elsewhere. Again, this distracts the 
examining attorney. 

Upon receiving a message, the examining attorney must 
5 decide when to read it, and, once read, must decide whether and 
when to use it. If prior to reading the message, the examining 
attorney asks the question contained therein, the message becomes 
no more than a distraction. If the examining attorney determines 
that the message contains a good question, the examining attorney 
10 either asks that question immediately and discards the note or 
saves the note for a later time. At times, an attorney may have 
ten to fifteen notes in front of him that he must deal with. 
This is particularly the case where the associate, knows more 
facts about the witness than does the examining attorney, or 
15 where the associate attorney is the more senior or more 
experienced of the two. 

. As previously, stated, the defending attorney must also take 
summarizing notes regarding both the Q & A' s in the deposition. 
From these notes, the defending attorney attempts to deduce 
20 upcoming lines of questioning, specific details of the examining 
attorney's positions, strengths . or weaknesses associated with 
those positions and objections such as "asked and answered" or 
"mischaracterizing the testimony" . These notes also remind the 
defending attorney to discuss incorrect statements made by the 
25 witness during a break so that therecord can be corrected when 
the deposition resumes. 

Among a variety of other duties also detracting from his 
note taking ability, the defending attorney must object whenever 
the question is inappropriate and state the reasons for his 
30 objections. Not only do these objections address evidentiary 
concerns, but some attorneys use the objection to warn the 
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witness to scrutinize critical questions posed by the examining 
attorney. as previously described. Furthermore, all such 
objections must be seasonable, i.e., before the witness answers 
a pending objectionable question. Note taking tends to detract 
5 from the defending attorney's ability to seasonably object and 
vice versa . 

To help alleviate this problem, the defending attorney 
often brings along an associate defending attorney. Much like 
the associate examining attorney, the. associate defending 
10 attorney takes notes and passes messages to the defending 
attorney. If the note involves making an objection, the 
defending attorney will rarely be able to make it seasonably. 
The messages just cannot be drafted, passed and read fast enough. 
;;The notes taken do however help fill the gap found in the 
15 defending attorney's notes who can then, focus on his other 
duties. In addition, the description of many of the other 
..problems encountered by the examining attorneys are equally 
applicable here . 

During a trial proceeding, both the plaintiff's attorneys 
20 and defendant's attorneys take turns assuming the role and duties 
of the .examining and defending attorneys as previously described 
• in the deposition proceedings. In many cases, instead of a 
single associate attorney, many associate attorneys may 
participate. The examining attorney at trial is called the 
25 "first chair". The most senior associate attorney, if at least 
one, is called the "second chair" and so on. There may be 
several "chairs" representing a single client. 

The problems described previously relating to a deposition 
directly, and perhaps more severely, apply to a trial proceeding. 
30 This is due in part to the increased number of attorneys 
involved. Additionally, several different problems associated 
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specifically with the trial proceeding also exist. For example, 
because the first chair is often on his feet and not near the 
other chairs on his team, passing messages or note taking proves 
to be very difficult if not impossible. This communication gap 
5 manifests other problems when the first chair attempts to obtain 
trial exhibits or other supporting materials. Scrambling to 
draft and pass a message to, or to reach whispering range of, a 
lower number ' chair often occurs. Other chairs that could 
possibly know of the location of the requested material are often 

10 oblivious to the request. 

Another communication gap exists between the attorneys at 
. trial and those that cannot attend. . Only those attorneys present 
and listening can respond at the proceeding. Non-attendees who 
might otherwise be able to assist have no means to evaluate the 

15 trial or communicate with the attending attorneys . 

On the defending attorney's side, the nomenclature using 
the first chair, second chair and so on is also used. The 
problems applicable to the defending attorneys during a 
deposition are also applicable and compounded at trial because 

20 of the enhanced significance of the defending attorney's duties 
beyond those of detailed note taking. As a result, much of the 
note taking must be left to other chairs. 

A judge must monitor, evaluate, and moderate the procedures 
and substance of the trial proceedings. Each question or line 

25 of questioning, answer and exhibit must be carefully analyzed for 
content, evidentiary form and relevance so that rulings on 
related objections can be. immediately made. If a judge is 
distracted, he is faced with problems previously enumerated with 
having to have the court reporter read back from the record. The 

30 judge also takes notes relating to various Q & A's or statements 
made during trial, particularly when ruling on issues of law or 
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fact from the bench. Due to the multitude of duties facing the 
judge, the judge's note taking is also subject to many of the 
other previously mentioned problems. 

Currently, facing the foregoing problems are over thirty 
5 . thousand court reporters and hundreds of thousands of attorneys 
and judges in the United States alone. Hence, it would be highly 
desirable to solve the foregoing variety of problems enumerated 
above facing attorneys, judges, court reporters and witnesses in 
conducting legal proceedings such as a deposition or trial by 
10 using and manipulating testimony generated in real-time by an 
automatic transcription system. 

It is therefore an object of the present invention to 
provide a method and apparatus which will aid the examining 
. .attorney's use and manipulation of testimony generated in real- 
■15. "time by an automatic transcription system. 

It is another object of the present invention to provide a 
; method and apparatus which aids associate examining attorneys in 
the use and manipulation of testimony generated in real-time by 
ah automatic transcription system to assist the examining 
20 attorney. 

It is yet another object of the present invention to 
provide a method and apparatus which aids the defending attorney 
in the use and manipulation of testimony generated in real-time 
by an automatic transcription system. 

25 It is another object of the present invention to provide a 

method and apparatus which will aid the associate defending 
attorney's use and manipulation of testimony generated in real- 
time by an automatic transcription system in assisting the 
defending attorney. 

30 It is therefore an object of the present invention to 

provide a method and apparatus which will aid the examining*. first 
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chair's use. and manipulation of testimony generated in real-time 
by an automatic transcription system. 

It is another object of the present invention to provide a 
method and apparatus which will aid the examining second chair's 
5 use and manipulation of testimony generated in real-time by an 
automatic transcription system in assisting the examining first 
chair . 

It is another object of the present invention to provide a 
method and apparatus which will aid all examining higher chairs' 
.10 use and manipulation of testimony generated in real-time by an 
automatic transcription system in assisting the examining second 
chair. 

It is yet .another object of the present invention to 
provide a method and apparatus which will aid the defending first 
15 chair's use and manipulation of testimony generated in real-time 
by an automatic transcription system. 

It is another object of the. present invention to provide a 
method and apparatus which will aid the defending second chairs' 
use and manipulation of testimony generated in real-time by an 
20 automatic transcription system in assisting the defending first 



It is another object of the present invention to provide a 
method and apparatus which will aid all defending higher chairs' 
use and manipulation of testimony generated in real-time by an 
25 automatic transcription system in assisting the defending second 
chair. 

It is. a. further object of . the present invention to provide 
a method and apparatus which will aid the judge's use and 
manipulation of testimony generated in real-time by an automatic 
30 transcription system. 
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SUMMARY OF THE INVENTION 

These and other objects of the present invention are 
achieved in a transcription network having a reporter's terminal, 
an attorney's terminal, and an associate's terminal with a 
5 communication link therebetween. Specifically, the attorney's 
terminal and associate's terminal receive text signals 
representative. of spoken words from the reporter's terminal via 
the communication link, and display the text. The associate's 
terminal also provides a means for generating and sending 
10. messages via the communication link to the attorney's terminal. 

Other objects are also achieved in a transcription system 
having a terminal, for transcribing signals representative of 
spoken words into corresponding alphabetic and numeric text, 
.which is linked to a recorder for recording spoken words. A 
15 means which operates across the link creates associations between 
recorded spoken words and corresponding alphabetic and numeric 
. :. text . 

Objects are also achieved in a reporting system having a 
transcriber node which operates on coded representations of 

20 spoken words and having cross-reference and phoneme libraries. 
Specifically, the reporting system uses the cross-reference 
library to identify associations between coded representations 
of spoken words and corresponding alphabetic and numeric text. 
Where associations cannot be identified, the reporting system 

25 then uses a phoneme library for associating spoken words with 
pronounceable text . 

Objects are also achieved in a reporting system operable on 
coded representations of spoken words. Particularly, the 
reporting system comprises a transcriber which cross-references 

30 the coded representations of spoken words so as to identify 
corresponding alphabetic and numeric text. The identified text 
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is then analyzed and, if appropriate, objections to the spoken 
words are provided. 

Objects are also achieved in a transcription network 
comprising a plurality of terminals each having a display screen 
5 which is electrically controllable for visually displaying * 
alphabetic and numeric text. A hierarchical communication link 
controls the communication flow between the plurality terminals. 
Each of the terminals send and receive messages via said 
hierarchical communication link, which provides for selective 
10 relaying of . messages between said plurality of terminals. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a perspective view of an embodiment of an overall 
system configuration according to the present invention. 

Fig. 2 is a schematic block diagram of an embodiment of the 
5 stenographic recorder used in the present invention. 

Fig. 3 is a detailed view of the interconnection of the 
stenographic recorder and CAT system. 

Fig. 4a is a diagram representing the association of data 
fields into a record which is used as a basis for the overall 
10 data structure for transcription according to the present 
invention . 

Fig. 4b is a detailed diagram representing the overall data 
structure used by the CAT system in transcription. 

Fig. 5a is a diagram representing the phoneme library used 
15 in the present invention wherein substitutions of readable 
phoneme text is found for untranslated key-strokes. 

Fig. 5b is an exemplary diagram of a foreign language table 
which is used to cross-reference text of a first language that 
corresponds to translated words of a first language. 
20 Fi 9- 6a is a perspective view of the examining attorney 

terminal and its interconnection to the CAT system and 
stenographic recorder. 

Fig. 6b is a perspective view of the examining attorney 
terminal's which provides further detail for both the display and 
25 keyboard. 

Fig . 7a' is a perspective view of the examining attorney 
terminal displaying the outline mode of the present invention 
which provides text editing of a case law outline indexing a set 
of preset messages. 
30 Fi 9- 7b is a perspective view of the examining attorney 

terminal displaying a tree-structured outline providing 
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hierarchical display of the categories of law to be investigated 
during a deposition. 

Fig. 8 is a perspective view of the associate examining 
attorney terminal which provides details in both the display and 
5 keyboard and its interconnection to the examining attorney 
terminal,. CAT system and stenographic recorder. 

Fig. 9 is a perspective view illustrating the use of audio 
or video tracking in the present invention. 

Fig. 10 is a perspective view of the wireless and hardwired 
10 interconnection of an alternate embodiment of the transcription 
system in a court room setting. 

Fig. 11 is a perspective view of another embodiment of the 
present invention which cooperatively utilizes stenographic and 
voice transcription. 
15 Fig. 12 is a diagram illustrating two alternate variations 

of the present invent ion. where the court reporter terminal is not 
linked to the transcription network. 

Fig. 13 is a perspective view of another embodiment of the 
present invention where, instead of using a keyboard, the 
20 attorney terminals 15 and 17 utilize pen-based or touch- sensitive 
screen input . 

Fig. 14 illustrates a flow-chart describing. the 
communication protocol used by the attorney terminals for 
communicating with the reporter terminal (or CAT system) 21. 
25 Fig. 15 illustrates an alternate flow-chart describing the 

reporter terminal's functionality in carrying out the Q&A 
communication protocol. 
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DESCRIPTION OF THE PREFERRED EMBODIMENT 



Fig. 1 is a perspective view of an embodiment of an overall 
.transcription system configuration according to the present 
invention. A stenographic recorder 11 is used by a court 
5 reporter at a deposition, hearing or other transcription matter, 
to record digital coded signals representative of verbal 
communications as they occur in real-time. Using a communication 
link 19, the recorder 11 transfers the representative signals to 
a computer aided transcription ("CAT") system 13, a computer 
10 terminal used by the court reporter, for transcription into 
alphabetic and numeric text corresponding to the actual words 
spoken. 

The CAT system 13 communicates the alphabetic and numeric 
text it generates ("transcribed text") along two independent 
15 communication links 20 and 21 to an examining attorney's terminal 
...... 15 and a defending attorney's terminal 16, respectively. The 

examining attorney's terminal 15 relays the text along a 
communication link 23 to an associate examining attorney' s 
terminal 17. Likewise, the defending attorney' s terminal 16 
20 relays the communicated alphabetic and numeric text representing 
the spoken words along a communication link 22 to an associate 
examining attorney's terminal 18. Upon receipt of the 
communication, the attorney's terminals 15, 16, 17 and 18 display 
the alphabetic an numeric text and provide a variety of tools for 
25 reviewing and evaluating what has been received. Relaying of 
transcribed text is implemented for low cost RS-232 and/or 
parallel configurations. If a more robust communication network 
is used, such as ethernet, the CAT system 13 routes the 
transcribed text directly to each terminal in the network. 
30 Identification of which type communication network is in place 
occurs via the command lines of the attorney's terminals. 
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Concurrent with receiving and displaying representative 
text,, the examining terminals 15 and 17 provide a vehicle for the 
examining and associate examining attorneys to exchange messages. 
The message exchange.- occurs through the communication link 23. 
Similarly, message exchanges between the defending terminals 16 
and 18 are provided for via the communication link 22. 

As illustrated, the CAT system 13 may be positioned for 
easy access on a conference room table. 25 in front of the court 
reporter. Also on the table 25, the terminals 15, 17, 16 and 18 
may be found positioned in front of an examining attorney, 
associate examining attorney, defending attorney and associate 
defending attorney, respectively. Each of the terminals 15, 16, 
17 and 18 and the CAT system 13 (the reporter's terminal) all 
include keyboards 27 and a screen 29. Each screen is flat or 
planar so as to prevent viewing by the opposing attorneys (and 
witness) who typically sit on either side of the table, as shown. 

Although preferred,, neither the keyboard nor the screen are 
necessary for the CAT system 13. In fact, the terminal itself, 
i.e., the functionality thereof, might exist within other nodes 
on the transcription network. For example, the functionality of 
CAT system 13 might be fully distributed between the recorder 11 
and the examining attorney's terminal 15. The functionality 
might also be located at some remote, off -site location. 
Similarly, the present invention contemplates many situations 
where terminals are not available for each attorney present, such 
as where only the examining side uses terminals or visa versa, 
or where either side utilizes a single terminal. Situations may 
arise where no attorneys possess a terminal. 

The use of additional terminals is also contemplated by the 
present invention.. Although- not shown in Fig. 1, a sixth 
terminal might be used: 1) for the witness who sits to the left 
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of the defending attorney on the defending attorney's right side; 
2) off -site for a. magistrate or judge; 3) off -site for a client 
or expert witness; or 4) off or on site for additional attorneys 
on the case. These configurations and their advantages and 
5 features will be discussed in detailed below. 

Fig. 2 is a schematic block diagram of an embodiment of the 
stenographic recorder 11 used to generate digitally coded 
stenographic signals representing the voice transcript. Recorder 
11 includes a keyboard 31 which is actuated by the court reporter 
10. in response to the spoken word. The reporter strokes the 
keyboard by actuating either one key individually or several keys 
simultaneously to represent phonetically the spoken word or some 
phonetic portion thereof. Such a key-stroke causes print 
actuator 33 to print a single line of type representing that key- 
15 stroke onto a paper tape 35. Tape 35 is thereafter incrementally 

— scrolled out in preparation for the next key-stroke. 

During the deposition, the court reporter may be asked to 
read back testimony. The reporter does so by sequentially 
viewing each line of phonetic characters typed onto paper tape 
20 35, mentally translating a single or several lines of characters 
into words, and then speaking out the phonetic translation. An 
attorney, judge or witness typically cannot read the phonetic 
characters typed onto tape 35. L 

In addition to being printed on paper tape 35, each key- 
25 stroke is converted to a digital code which is stored in a random 
access memory (RAM) 37 and also transmitted by a communication 
port 39 to CAT system 13 via a serial output bus 41 and 
communication link 19 (Fig. 1). if print actuator 33 runs out 
of tape 35, stenographic recorder 11 will still store and 
30 transmit the digital key-stroke codes. Similarly, if 
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communication link 19 is disconnected, storage and printing will 
still occur. 

More specifically, a keyboard scanner 43 monitors keyboard 
31 via scanning bus 45. Upon each stroking of keyboard 31, 
5 keyboard scanner 35 responds by generating a digital key- stroke 
code representing all of the keys pressed in that stroke. Next, 
keyboard scanner 43 interrupts a central processing unit ("CPU") 
47 via an interrupt line. 49. CPU 47 responds to the interrupt 
by placing the address of keyboard scanner 43 upon an address bus 

10 51. In response, an address decoder 53 generates a select signal 
on a scanner select line 55 which enables keyboard scanner 43. 
CPU. 47 then places a read enable signal on a read line 57. 
Keyboard scanner 43 responds by placing the digital key-stroke 
code on a data bus 59. Next, CPU 47 reads the key-stroke code 

15 into an internal key-stroke register 61 and increments a print 
count register 63, a transmit count register 65 and a memory 
pointer register 67. The count in register's 63 and 65 
represents the back-log of key-stroke codes to be printed and 
transmitted, respectively. Immediately after the key- stroke code 

20 has been read, keyboard scanner 43 removes the interrupt and 
awaits the court reporter's next stroke. 

Having retrieved the key-stroke code, CPU 47 must first 
save the code and then attempt to transmit .and print it. 
Specifically, from the contents of memory pointer register 67, 

25 CPU 47 computes the next available address in RAM 37 and places 
that address onto address bus 51, while placing the contents of 
key-stroke register 61 onto data bus 59. In response to the 
address on address bus 51, address decoder 53 produces a select 
signal on RAM select line 69, selecting RAM 37. Afterwards, CPU 

30 47 places a write enable signal upon a write line 71, and RAM 37 
responds by storing the key-stroke code. 
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Having saved the key- stroke code, CPU 47 enters a 
prioritized wait-mode, awaiting an interrupt signal from: 1) 
keyboard scanner 43 along interrupt line 49 , signifying that 
another key- stroke code has: been detected and generated; 2) 
5 communication port 39 along interrupt line 73, signifying that 
port 39 is ready to receive a key- stroke code for transmitting; 
or 3) print actuator 33 along interrupt line 75, signifying that 
actuator 33 is ready to receive another key- stroke for printing. 
Interrupt servicing priority is given to scanner 43, followed by 

10 port 39 and actuator 33. 

This servicing scheme does not provide first in first out 
servicing of interrupts. Instead, at the conclusion of servicing 
any interrupt, CPU 47 enters the prioritized wait-mode and, 
first, unmasks interrupt line . 49 from scanner 43. If an 

15 interrupt is present, CPU 47 exits the wait-mode and services 
scanner 43. Otherwise, with no interrupt from scanner 43, CPU 

- 47 examines the backlog count in registers 63 and 65. If neither 
contain a backlog, CPU 47 waits solely for an interrupt from 
scanner 43. 

20 If one or both contain a backlog, priority is given to the 

register with the largest backlog or to transmit count register 
65 if the registers are equal. With this priority, CPU 47 
unmasks the corresponding interrupt line. If the corresponding 
device has provided an interrupt signal signifying that it is 

25 ready, CPU 47 will exit the wait-mode and service that device. 
Otherwise, if that device is not ready, CPU 47 will unmask the 
final interrupt line. At this point, CPU 47 will service the 
first interrupt that occurs. Finally, before servicing any 
interrupt, CPU 47 masks all interrupts. 

30 Having described the prioritization scheme and the 

servicing of keyboard scanner 43 , we move on to servicing 
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communication port. 39. As previously detailed, during the 
servicing of scanner 43, transmit count register 65 contains a 
backlog count of at least one, signifying the need to transmit. 
The count in register 65. directly represents the binary number 
5 of key-strokes that have been stored but not transmitted. If 
register 65 contains a binary value of one at the time CPU 47 
responds with servicing, CPU 47 will place the address of 
communications port 39 onto address bus 51, while placing the 
contents of key-stroke register 61 (the recently received key- 

10 stroke code) onto data bus 59. Address decoder 53 responds by 
placing an enabling signal on port select line 77, selecting 
communications port 39. CPU 47. next places a write enable signal 
upon write line 71, causing the key-stroke code on data bus 59 
to be written to communication port 39. Upon receiving the key- 

15 stroke code, port 39 removes the interrupt signal from interrupt 
line 4 8. until the key-stroke code is transmitted to CAT system 
13 via communication link 19 (see Fig. 1). CPU 47 exits the 
servicing routine and enters the prioritized wait -mode. 

If, however, transmit count register 65 reveals a greater 

20 backlog than just one, CPU 47 must retrieve the oldest un-sent 
key-stroke code from RAM 37 before sending it to communication 
port 39 as described above . To retrieve the oldest un-sent key- 
stroke code, CPU. 47 calculates the address of the oldest key- 
stroke code by subtracting the contents of transmit count 

25 register 65 from memory pointer register 67 and placing that 
value on address bus 51. . Address decoder 53 responds by 
selecting RAM 37 via RAM select line 69. Next, CPU 47 provides 
a read enable signal on read line 57 which causes RAM 37 to place 
the oldest un-sent key-stroke code onto data bus 59. CPU 47 then 

30 reads this key-stroke code into an auxiliary register 79, 
completing the retrieval. Finally, the same steps described 
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above are engaged in by CPU 47 to write this key-stroke code 
(stored in auxiliary register 79) to communication port 39. 

In an identical niahner, responding to interrupt line 75, 
CPU 47 services print actuator 43 utilizing address decoder 53 's 
5 selection of actuator 33 (via actuator select line 81) and either 
print count register 63 or auxiliary register 79 as necessary. 
The details therefore need not be repeated. 

A read only memory ("ROM") 83 provides the instructions or 
program for CPU 47 which defines the operation described above . 

10 This program also defines the initialization and mode of 
operation of stenographic recorder 11. ROM 83 is accessed much 
the same way RAM 37 is accessed, using ROM select line 85 from 
address decoder 81. 

Additionally, although not shown, RAM 37 can be. backed-up 

15 with batteries, providing a relatively permanent storage of the 
record for later retrieval. Otherwise, stenographic recorder 11 
can service a disk drive system (not shown) which could either 
operate in parallel with RAM 3 7 but at a lower priority in the 
servicing scheme, or completely replace RAM 37. With either 

20 scenario, the disk drive system would create the permanent 
storage on a diskette. 

As a final. matter, while in the prioritizing wait-mode, CPU 
47 can evaluate the backlog of the devices which are being 
serviced and read the status of a severely back- logged device to 

25 identify a problem. Warning messages can then be sounded or 
displayed to the court reporter. This might also be accomplished 
by using additional interrupt lines signaling malfunctions. 

Fig. 3 is a detailed view of the interconnection of 
stenographic recorder 11 and CAT system 13. As previous 

30 described, in response to the court reporter stroking keyboard 
31, recorder 11 creates a key-stroke code and transmits this to 
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the CAT system 13 (the reporter's terminal) along communication 
link 19. Key-stroke codes received are displayed in two 
different formats on a screen 100 of CAT system 13. In a tape 
window 101 which is shaped and sized much like paper tape 35, 
5 alphabetic representations for received key-stroke codes are 
displayed in a line by line fashion wherein each key-stroke code 
is represented on a single line. Tape window 101 thus provides 
an electronic version of a paper tape which is generally in the 
format familiar to a court reporter. 

10 Screen 100 also includes a transcription window 103 which 

displays the alphabetic and numeric text representative of the 
spoken words, i.e., displays the transcribed version of the key- 
stroke codes received from recorder 11. Through user selection, 
the windows 101 and 103 can be configured to update the screen 

15 100 upon complete transcription of each: 1) word; 2) sentence; 
3) Q & A; or 4) Q or A. The display can also be chosen to 
operate in either scrolling or paging fashion. For example, with 
a preferred display selection, the transcription window 103 will 
only display the current Q & A as it is developed on a sentence 

20 by sentence basis. After completing the transcription of the 
first sentence of a new question, the CAT system 13 clears the 
currently displayed Q & A (the old Q & A) from the window 103, 
and displays the transcribed alphabetic and numeric text of the 
first sentence of the new question. As each additional sentence 

25 of the new Q & A is received and transcribed, it too is added to 
the window 103. After fully transcribing the new QUA, window 
103 holds the display of that new Q & A until the end of the 
first sentence of the next Q & A; upon which, this cycle repeats. 
To set up the mode of updating the windows or. to access any 

30 of a variety of other CAT system 13 commands, the screen 100 also 
includes a command line 105. In one embodiment, the command line 
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105 lists several one-word commands displayed horizontally in a 
single line of text. . .The commands listed by the command line 105 
can be selected directly using a mouse, pen, fingertip or other 
input device (none of which are shown) or indirectly by 
5 simultaneous pressing the "alt", key and the first letter of the 
desired command on the keyboard 107. Any other mode or method 
of command selection involving, for example, button bars, pull- 
down menuing, or icons might also be used. 

As previously stated, a court reporter strokes the 

10 stenographic recorder 11 in response to the spoken word. 
Generally, a given word may demand either a single stroke or 
multiple strokes based on the complexity of its phonetic 
composition. Upon completing the strokes necessary for a given 
word, the court reporter immediately begins stroking the next 

15 word. No space bar or other word separator is stroked between 
words. As a result, all of the strokes run together. At the end 
of a sentence, the court reporter does stroke recorder 11, 
however. The CAT system 13 operates on this sequence of key- 
strokes to determine the beginning and end of each word. 

20 Providing the foundation of the transcription process, the 

court reporter uses a personalized cross-reference library which 
lists the readable text of every word that the court reporter has 
translated to date. Each of these words are associated with the 
corresponding stenographic key-stroke or sequence of key-strokes 

25 that represents that word. This cross-reference library can be 
displayed in a visual manner, but is stored in binary tree 
structure for efficient searching. 

Upon receiving the first key-stroke code of the sentence, 
CAT system 13 begins transcription. This first code is compared 

30 to all of the possible first stroke codes within the . court 
reporter's cross-reference library. If a complete cross- 
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referenced word is found, CAT system 13 preliminarily treats this 
as the entire word. Then, upon receiving the second key-stroke 
code in the sentence, CAT system 13 treats this second key- stroke 
code as the beginning of another word and looks again to all of 
5 the possible first stroke codes-within the library. If that code 
is not found, CAT system 13 must back-track. The first and 
second codes are combined and are then compared to all of the 
possible first and second stroke codes within the library. If 
a complete cross-referenced word is found, CAT system 13 replaces 

10 the shorter, single stroke word with this new, two stroke word, 
and preliminarily treats this as the entire first word. Next, 
a third code is received and is similarly treated as the 
beginning of another word, and the cycle repeats. 

By the end of the sentence, CAT system 13 provides the best 

15 transcription possible. Key-strokes that still remain 

untranscribed at that point are translated using a phoneme 
library into readable phoneme representations of those key- 
strokes to aid both the court reporter and, more importantly, 
those lay-persons down-line. By pronouncing these phonemes, the 

20 lay-person might identify the actual word or words which the 
cross-reference library did not contain. Substituting the 
phoneme representation with the actual word or words causes the 
cross-reference library to be updated. 

Once fully transcribed using the cross-reference and 

25 phoneme libraries, the entire sentence is communicated down- line 
to attorney terminals 15 and 16 via communication links 21 and 
20, respectively. 

More specifically, Fig. 4a is a diagram representing the 
association' of data fields into a record which is used as a basis 

30 for the overall binary tree, data structure used in transcription 
according to the present invention. In particular, CAT system 
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13 utilizes a linked-list arrangement of two types of data 
records: a. key-stroke code listing (KCL) record 151 and a 



provide the storage structure for the court reporter's 
5 personalized cross-referencing library. 

Basically, CAT system 13 uses records 151 and 153 to 
associate each individual key-stroke code with as many subsequent 
key-stroke codes to represent a given transcribed word. KCL 
record 151 associates: 1) a listed key-stroke code (LKC) field 
10 155/ for storing a specif ic key-stroke code; 2). a listing 
optimizing counter (LOC) field 157, for storing a count used for 
optimizing future searches; 3) a first KCL_record pointer field 
159, for storing a pointer to the next KCL record on this level; 
4) a CT record pointer field 161, for storing a pointer to an 
15 associated CT record; and 5) a second KCL_record pointer field 
-.--163, for storing a pointer to a corresponding KCL_record at the 
j: next listing level down. Similarly, CT record 153 associates: 
1) a CT string field 165, for storing a string of text; 2) a text 
optimizing counter (TOC) field 167, for storing an additional 
20 count used for optimizing transcription; and 3) a CT record 
homonym pointer field 169, for storing a pointer to another CT 
record containing a homonym to the contents of CT string field 
. 165. 



25 overall binary, linked-list data structure used by CAT system 13 
in transcription. KCL records 200, 201 and all KCL records (not 
shown) directly to the right and left of records 200, 201 
constitute a first listing level. This first listing level is 
no more than a linked-list of the each beginning stroke in the 

30 stenographic representation of the words held in the court 
reporter's library. Upon receiving the first code of a sentence, 



corresponding text (CT) record 153. These two types of records 



Fig. 4b is a detailed diagram representing a portion of the 
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that code is compared with each code stored in each KCL record 
on the first listing level. If, for example, the received code 
does not match the stored code in KCL record 200, KCL record 200 
points to the next KCL record 201 for a similar comparison to the 
code stored therein. In this. manner, by stepping through the 
first listing level, a matching KCL record can be found. 

Assuming that the code stored in KCL record 200 does match 
the first key-stroke code received, the associated CT records 205 
and 207 can be accessed to retrieve readable cross-referenced 
text. KCL record 200 can also be used as a back-tracking point. 
If the word transcribed requires two key-strokes, KCL record 200 
points to the second listing level . This second listing level 
is specifically associated with KCL record 200 and begins with 
KCL records 202 and 203 followed by all KCL type records (not 
shown) to the right of record 203. Any second code received 
which follows a first, code which matches that stored in KCL 
record 200 is compared to the codes stored in the KCL records on 
the second listing level. KCL record 204 represents yet a third 
listing level under, the key-stroke sequence stored in record 200 
and 203, and so on. CT records may or may not be associated with 
a given KCL record. KCL record 202 exemplifies such a situation. 
Additionally, only a single CT record is generally associated 
with a single KCL record, such as is shown with KCL record 203 
and CT record 209. Only when homonyms exist will there be 
multiple CT record association as shown with KCL record 200 and 
CT records 205 and 207. .- 
Particularly, : upon receiving a first key-stroke code from 
stenographic recorder 11, CAT system 13 begins a transcription 
expedition by parsing through a first listing level in an attempt 
to find a matching KCL record. If a matching KCL record is found 
which has an associated CT record transcription, CAT system 13 
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decides there is a preliminary transcription and treats the 
second (next) code received as the beginning of a new word, 
parsing. the first listing level for a matching KCL record. 

If a matching KCL record is found for first code received 
5 yet without an associated CT record transcription, the second 
code received is compared for a matching KCL record at the second 
listing level, if a second level beneath the matching KCL record 
on the first level exists. If a match is found at the second 
listing level with an associated CT record transcription, the 

10 third key-stroke code is treated as the beginning of a new word, 
and parsing at the first listing level repeats. 

If after a match with associated transcription text a key- 
stroke yields a dead end, back-tracking must occur. The requires 
picking up where the last successful parsing expedition left off, 

15 searching deeper listing levels to see if what had been 

considered an entire word is really only a portion thereof. If 

a match is found with an associated CT record transcription, that 
transcription at that deeper listing level is stored, and the 
next key-stroke code received is treated as the beginning of a 

20 word and the cycle repeats. 

With each successive, unsuccessful parsing round, the 
previously described transcription process becomes more and more 
complex with many parallel and nested transcription pathways 
being considered. If available, the first completely .transcribed 

25 sentence found is communicated to attorney terminals 15 and 16. 

If a completely transcribed sentence is unavailable, a 
second phase of transcription, the phoneme transcription phase, 
is invoked. In this phase, as illustrated by Fig. 5a, each 
untranscribed key- stroke is cross referenced into a phoneme 

30 replacement table or phoneme library 220 to obtain readable 
phoneme text. Specifically, the CAT system 13 parses through a 
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series of listed key-stroke codes held in a listed_keystroke_code 
field 223 of the phoneme library 220 to locate the key-stroke 
code entry which corresponds to the untranscribed key-stroke 
code. Once located, the CAT system 13 extracts the readable 
5 phoneme text stored in a corresponding_j?honeme field 221 of the 
phoneme library 220. Thereafter, extracted text is substituted 
for the untranscribed key- stroke in the sentence. Once all of 
the untranscribed key- strokes of the sentence have been replaced 
with corresponding phoneme text, the sentence is communicated 

10 down-line to the attorney terminals. For example, if the CAT 
system 13 receives a key-stroke code representing the key-stroke 
» W A Z" for which no counterpart exists in the 

cross-reference library, the CAT system 13 parses the listed_key- 
stroke_code fields 223 and locates the specific entry in a field 

15 225. The corresponding readable phoneme text entry "was" in a 
field 227 is then extracted and substituted for the untranscribed 
key-stroke code in the sentence. 

In addition, if. homonyms are found, signifying multiple CT 
record association, they are displayed within parenthesis. For 

20 example, the homonyms "their", "there" and "they're in the 
question "Were you there?" would be displayed as: 

Q. Were you (there | their j they' re) ? 
Although readable, to correct such common homonym transcription 
problems, another level of transcription is contemplated using 

25 so-called "artificial intelligence" techniques for determining 
the correct word grammatically in the context of the given 
sentence. 

As a final matter associated with the data structure, 
optimizing counters are placed in each record so as to provide 
30 optimization of the overall structure. Each time a record is 
used in the final translated sentence, the associated optimizing 
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counters are incremented. At the end of a deposition, the court 
reporter can run an optimizing routine which sorts records in 
each linked list based on the count in the optimizing counters. 
In this way, the most common matching KCL records are encountered 
5 first, speeding up the parsing response time. similarly, the 
most common homonyms will appear first in the parenthesis. 

Referring back to Fig. 1, GAT system 13 communicates a 
sentence down-line to terminals 15,16,17,18 when that sentence 
has been fully transcribed. Full transcription takes place 

10 whenever CAT system 13 receives an. end of sentence key- stroke 
code ("EOS") from stenographic recorder 11. If, however, while 
in mid-sentence, the witness pauses to think, an EOS will not be 
received. To satisfy, an attorney who wants to take this 
opportunity to review what has been stated thus far, an 

15 artificial EOS can be generated. An artificial EOS is generated 

- - if either: 1) the pause between the court reporter's key-strokes 

- ' exceeds a preset time limit (programmable from zero to one 

hundred and twenty seconds) ; or 2) an attorney down-line presses 
the space bar on terminal 15, 16, 17 or 18 indicating via the 

20 corresponding communication links 20,21,22,23 the desire to 
display what has been received thus far. In response to an 
artificial EOS, CAT system 13 completes the translation as if an 
EOS has been received, attaches an artificial EOS to the end, and 
sends the transcribed "partial" sentence down-line. Thereafter, 

25 if the EOS is the very next key-stroke received, CAT system 13 
merely communicates an EOS down-line to replace the artificial 
EOS. Alternatively, if further non-EOS. key- strokes are received 
before receiving an EOS, CAT system 13 completes the 
transcription of the entire sentence with the EOS and sends it 

30 down-line. Terminals 15,16,17,18 over-write the "partial" 
sentence with the complete sentence, received. In the 
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alternative, terminals 15,16,17,18 might instead merely display 
partial sentences, only writing full sentences to file. 

Depositions involving participants in two or more languages 
are also contemplated by the present invention. In a first 
5 embodiment, the transcription process operates as previously 
described by providing a word for word transcription of the 
spoken question in a first language. Each of these words are 
then cross-referenced into a. second language table to generate 
a sentence in a second language. Although the use of. the second 

10 language table can be assigned to each attorney terminal down- 
line, in a. preferred mode,, the CAT system 13 performs the task. 
Once translated, the CAT system 13 transmits the second language 
sentence down- line to viewing participants who understand that 
second language. By using additional language tables, the CAT 

15 system 13 can provide translations for as many languages as is 
needed. 

If the CAT system 13 performs the translation task, the CAT 
system 13 also manages the specific down- line routing of first 
and second language sentences, automatically. To accommodate 

20 automatic routing, all participants are required to identify 
their viewing language and their role in the deposition upon 
logging in to the system 13. The CAT system 13 stores the 
identified information in a log-in list. By referencing the log- 
in list, the CAT system 13 automatically configures itself so as 

25 to provide each participant with a translation in. their 
appropriate language. 

An exemplary second or foreign, language table 231 is 
illustrated in Fig. 5b. Using the language table 231, the CAT 
system 13 parses the text entries of a first language in a series 

30 of identif ied_CT_String fields 233 to locate the transcribed text 
of a word in a first language. Once located, the text of a 
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second language is extracted from a corresponding_translation 
f ield 231 and placed into a second * language sentence . Each 
transcribed word in the first language undergoes this process so 
that a second language sentence can be fully constructed. 
5 Thereafter, both the first and second language sentences are 
transmitted down-line to the appropriate participant. For 
example, the word "good" located in a field 235 in a first 
, language, English, would be directly substituted by the word 
"bueno" of a second language, Spanish, contained in field 237. 

10 Although translations would not always be exact because of word 
context, the use of a human, translator could be minimized. 
• Moreover, as a supplement to the language table 23 0, word context 
analysis might also be implemented to increase the accuracy of 
the translations. 

15 If the down-line participants that speak during the 

deposition all share a common language, the court reporter need 
; only understand the common language and provide a cross-reference 
library and a phoneme library for that language. Alternate 
language translations for non- speaking, down-line participants, 

20 are provided using as many different foreign language tables as 
needed, as described above. Routing and language information is 
retrieved from the log- in list. 

If, however, the down-line participants that speak during 
the deposition share two or more languages, in a preferred 

25 embodiment, a multi-lingual court reporter provides a cross- 
reference library and phoneme library for each of the languages 
being spoken. Additionally, for each spoken language, language 
tables providing translation into each of the multiple languages 
are also necessary. During the translation process, when the 

3 0 multi- lingual court reporter indicates the beginning of a 
question or answer, or otherwise identifies the speaker, the CAT 
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system 13 automatically selects the appropriate cross-reference 
and phoneme libraries to transcribe the spoken language, then 
accesses the appropriate foreign language tables to construct 
sentence translations. The resulting translations are 
5 automatically routed down-line. The CAT system 13 performs the 
automatic selection of libraries and tables and routing with 
reference to the entries in the log-in list. 

Although the use of multiple phoneme libraries is preferred 
in a multiple language deposition, they may be completely 

10 ignored. Otherwise, when attempting to translate phoneme text 
into a second language, the phoneme text should be replaced by 
equivalent phoneme text in the second language. Therefore, 
pronunciation of the spoken word would sound the same in either 
language. Because with phoneme text the actual transcription of 

15 a word is unknown, a second language translation cannot be found. 
Therefore, as a substitute, a. pronunciation for each of the 
participants in the phonemes of their language which sounds like 
the spoken word in the first language is provided. 

Referring to Fig. 6a, examining attorney terminal 15 

20 includes a screen . 251 and a keyboard 253. Screen 251 is split 
into a transcription window 255 and a communication window 257, 
having a common command line .259. The display of window 255 is 
very similar to that of window 103 in CAT System 13 {Fig. 3), 
displaying the transcribed text and having the same setup 

25 response choices. This transcribed text is received by terminal 
15 from CAT System 13 in coded form along communication link 21 
in a sentence by sentence, fashion as described above. 

Even though transcription is received on a sentence by 
sentence unit basis, in the preferred mode of response, the 

30 sentences units received are not displayed in window 255 until 
an entire question or responding entire answer has been received. 
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Three indicators are used to signify the end of a question (EOQ) 
or end of . an answer (EOA) : 1) a court reporter's key-stroke 
signifying the beginning of an answer,, clearly indicating an EOQ 
and visa versa; 2) the expiration of a preset delay (programmable 
5 from zero to one hundred and twenty seconds) since the last 
received sentence, providing an artificial EOQ or EOA; and 3) the 
examining attorney pressing a space bar 261, indicating a desire 
to see what has been received thus far and providing an 
artificial EOQ or EOA. Because the second and third indicators 

10 only signify that a question or an answer has possibly ended, 
they are updated upon receiving a true EOQ or EOA. 

As illustrated in Fig. 6b, the question begins in window 
255 with the symbol M Q" followed by the question.. For example, 
Fig. 6b shows the display: 

15 Q. When did that occur? 

Afterwards, upon receiving the entire transcribed answer 

(signified by an EOA or an artificial EOA), the answer appears 
in window 255 directly below the question. The answer appears 
on window 255 following the symbol "A". For example, Fig. 6b 

2 0 shows the answer: 

A. Around noon. 

If the remaining room in window 255 is such that the Q & A 
cannot be simultaneously displayed, the answer is displayed at 
the bottom-most portion of window 255 while the remaining room 
25 is allocated to the end of the question. 

Each Q & A remains displayed until the next transcribed 
question is received in its entirety. This is defined by either 
the receipt of an EOQ or an artificial EOQ. Thereupon, the last 
Q & A is cleared from window 255 and the entire next question is 
30 presented at one time, and the Q & A cycle repeats. 
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By permitting only the display of one Q & A unit at a time, 
transcription window 255 remains uncluttered, and the examining 
attorney can easily focus on the pending question and\or the last 
answer. Thus, for each successive Q & A unit, window 255 is 
5 cleared and then displays the full, transcribed text of the 
question followed by its corresponding answer. 

In a preferred mode, each Q & A unit is associated with a 
different number so as to provide a reference to specific Q & 
A's. The numbers are given in consecutive order for ease of 
10 ; reference. As shown in. Fig. 6b, the Q & A displayed is 
associated with the number 57, signifying that the displayed Q 
& A is the 57th Q & A received. In its entirety, window 255 
shows : 

* 57 Q. When did that occur? 

15 57 A. Around noon. 

Keyboard 253 includes an up arrow key 263 and a down arrow 
key 265 for scrolling through the Q & A's. For example, stroking 
the up arrow key 263 causes scrolling upwardly in a line-by-line 
fashion bringing the previous Q & A into display on window 255. 

20 For faster movement between Q & A's, keyboard 253 provides a page 
up key 267 and a page down key 269. In response to a single 
stroke, keys 267 or 269 cause paging between entire Q & A's. In 
addition, keyboard 253 includes a "home" key 270 and an "end" key 
271. No matter which previous Q & A is currently being 

25 displayed, actuation of end key 271 causes window 255 to display 
either the last full Q & A or, if no answer has been completed, 
to the last pending Q. Similarly, home key 270 causes window 255 
to display the first Q & A of the proceeding. 

Keyboard 253 also includes a "mark" key 273 which permits 

30 any Q & A to be marked for later reference. The marked Q & A is 
indicated in the window by an asterisk symbol appearing on the 
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screen to the left of the Q symbol as shown in Fig. 6b. 
Associated therewith, keyboard 253 also includes, a "quick-up" key 
275 and a "quick -down" key 277 which permit scrolling only 
through the asterisked marked questions. 
5 In particular, if an important answer or definition has 

been given by the witness, the examining attorney can mark the 
Q Sc A with an asterisk. Whenever the attorney wishes to refer 
back up to that answer, he or she may merely press quick-up key 
275 to recall to the display window 255 the marked questioned 

10 with its answer. The. attorney may also desire to refer back up 
to a Q & A marked several marked Q & A's ago. To accomplish 
this, successive strokes to quick-up key 275 pages backward 
through previously marked Q & A's. Although stroking end key 271 
returns the display in window 255 to the pending question, the 

15 examining attorney may desire to sequentially page down through 
- — the marked Q & A's as they occurred. The quick^down key 277 
provides this ability. In addition, marked Q & A's can be 
referred to after the deposition in preparation for the next 
day f s deposition. As understood, pertinent quest ions may also 

20 be marked at a time outside of the deposition, for example, later 
that evening at the hotel room before the next day's deposition. 

Accessing a specific Q & A can be achieved through the 
associated number assigned to that Q & A. By pressing a "number" 
key 278 followed by the number previously associated with a 

25 specific Q & A, terminal 15 locates and displays that Q & A in 
window 255. A specific Q & A can also be located through a 
lexical scanning capability accessible through a scan-up key 279 
and a scan-down key 281. For example, by pressing scan up key 
279 followed by a lexical searching template, attorney terminal 

30 15 scans backward through the previous Q & A' s in an attempt to 
find text matching the lexical template. If a match is found, 
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that Q & A is displayed on window 255, else a "no match found" 
message is received. Afterwards, stroking scan-up key 279 again 
pulls up the most recently used lexical template. By merely 
stroking scan-up key 279 without modification to the template 
5 continues the search further upward through previous Q & A's. 
Scan down key 281 works in a similar manner but scanning in the 
opposite direction. 

In general, the lexical template format and features are 
similar to those provided in legal searching databases familiar 

10 to most attorneys.- These templates i can be constructed using 
logical operators, wild card characters, quotes,, parenthesis and 
brackets with specific text to be located. The lexical template 
ordinarily must be satisfied by a single Q & A. To over-ride 
this "default" condition, a delimiter must be used.. For example, 

15 placing a "/A" or a "/Q" delimiter within the template, directs 
attorney terminal 15 to satisfy that template, or portion 
thereof, entirely within a single answer or a single question, 
respectively. A "/QA2 " delimiter expands the search to within 
two consecutive Q & A's; while a "/QA3" corresponds to within 

20 . three consecutive Q & A's, and so on. Similarly, a delimiter of 
. 11 /A3" or a delimiter of "/Q5" respectively require satisfaction 
of the lexical template to within three consecutive answers or 
within five consecutive questions. Delimiters "/W4", "/W5Q" , or 
"/W3A" require the respective satisfaction within: 1) four 

25 consecutive words in either a single Q or A; 2) five consecutive 
words in a question only; or 3) three consecutive words in an 
answer only. 

In addition to searching only the current transcript, 
lexical searching can also be applied to all other evidence in 
3 0 the lawsuit. The terminal 15 permits either locally or remotely 
stored transcripts recorded at previous hearings or depositions 
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to be selected and searched through the command line 259. 
Similarly, all other case evidence such as the pleadings 
documents or documents produced through discovery may be scanned- 
ih, converted to text using character recognition software, and 
5 made available for the lexical searching. Once located through 
a search, the scanned- in image can be retrieved and displayed or 
printed for use as an exhibit at the ongoing deposition. 

Remotely located legal databases such as Lexis® or Westlaw® 
can also be accessed and searched through the command line 259 

10 during the deposition. Searching other sources may be used to 
identify discrepancies in the witness's, current testimony, or to 
help clarify the law in an unexpected area so that appropriate 
questioning can be directed. 

Although boolean logic is used for formulating lexical 

15 search requests, a natural language front end is also 
contemplated. The attorney terminals could provide for user 
preference, invoking the boolean logic or natural language front 
end upon request. The lexical formulation for a search is 
identical whether searching any of the case evidence or' legal 

20 databases, thus providing uniformity of searching. 

The specific placement of the various keys shown on 
keyboard 253 is not critical. In fact, the illustrative 
placement in Fig. 6b is merely set forth for ease of explanation. 
Additionally, the functionality of these keys may be accessed 

25 directly from the command line 259 as explained above. 

Pressing a tab key 285, or through direct selection with a 
mouse, pen or finger-tip, causes toggling between transcription 
window 255 and communication window 257. The communication 
window 257 provides a visual display in either a stack mode and 

30 an edit mode. As illustrated, in the stack mode, communication 
window 257 displays the first line of each communication 
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received. These communications originate from three sources; 1) 
the examining attorney's keyboard 253 personal notes "From: 
Self" or messages "To:" others; 2) the examining attorney 
terminal 15 -- through artificial intelligence ("AI") algorithms 
5 "From: AI" which convey potential objections or through scheduler 
messages; and 3) the other terminals on the network such as the 
associate examining attorney terminal 17 messages "From:" 
others. While in. the stack. mode, window 257 can display all of 
the communications from one source, all sources, or any 

10 combination of the three sources. 

To fully read any communication, the examining attorney 
merely selects the desired communication from the stack of 
messages illustrated in the window 257, causing the window 257 
to display of the full text of the communication in an edit mode. 

15 Communications can also be created, modified, deleted, copied, 
printed, or communicated to the other terminals, such as 
associate examining attorney terminal 17 via the communication 
window 257. 

In this embodiment, one side's message exchange along that 
20 side's communication link cannot be intercepted by the opposing 
attorney's not only because the communication links are isolated, 
but also because of the controls placed on the CAT system 13. 
The CAT system 13 can only consider relaying communications from 
one side to the other if a specific request therefor is received. 
25 Moreover, even if a specific request is received, the CAT system 
13 cannot relay communications unless specific permission has 
been given to receive such communications. In particular, an 
attorney on one side sends a communication to the CAT system 13 
with a specific request that the communication be relayed a 
30 recipient on the opposing side.. Once received, the CAT system 
13 complies to. the request only if the recipient on the opposing 
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side grants permission for the relaying. To grant or retract 
permission, each side sends a message to the CAT system 13, at 
log- in or at any subsequent time, indicating whether or not 
requests to relay messages from the other side should be honored. 
5 If not, the CAT system 13 responds to each request by sending a 
do-not -disturb (DND) message to the requesting terminal. If so, 
•the CAT system 13 responds by relaying the message from the 
sending side's communication link to the receiving side's link. 
In either the edit or stack display modes, communication 

10 window 257 interfaces with a mouse/ keyboard 253 and associated 
keys in much the same way as does transcription window 255. For 
example, up and down arrow keys 263 and 265 and page up and down 
keys 267 and 269 respectively permit scrolling and paging through 
the stack or text. However, the scan-up and scan-down keys 279 

15 and 281 operate in a slightly different way. In particular, as 
a default, lexical scanning only parses the currently displayed 

— stack, i.e., the currently selected communication sources. A 
global command delimiter n /G" overrides the default so that all 
communications from all sources can be scanned. The following 

20 use of delimiters can override the default: a "/M" for messages 
only; n /N" for notes only; n /AI n for AI objections only; or for 
combinations of these fields -- for example n /MN" for messages 
and notes only. 

The examining attorney may also choose to modify a 

25 significance indicator, preferably a fuzzy-logic type slide-bar 
icon, which causes the attorney terminals to associate a level 
of priority with each message, note, or other communication 
received. The significance indicator may default at an average 
or middle setting and need not be adjusted. If selected as a 

30 set-up option, the significance indicator can be used, along with 
the age of the communication, to adjust the order of the stack 
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of communications. The significance indicator can also be used 
to associate a window or window border color with the level of 
priority the communication carries. When set to a maximum value 
the significance indicator can also be used to send a flashing 
5 message to another terminal, indicating immediate attention is 
desired. This functionality exists on each of the terminals in 
the transcription network. 

The examining attorney may also type to himself his own 
notes as the deposition is progressing. These notes may serve 

10 useful in that they may be reviewed later that evening in order 
to prepare for the next day of deposition, or they may be used 
at a later time during the deposition when a question becomes 
ripe. Most attorneys take notes during, a deposition. Those who 
type proficiently may take notes with keyboard 253, and those who 

15 do not might use pen-based input as described more fully below 
in relation to Fig. 13 . . 

. Alternatively, the examining attorney may type or pen notes 
into his terminal before the deposition begins. In particular, 
the attorney terminals 15 and 17 provide an outline mode for 

20 hierarchically outlining by subject category and sub-category a 
series of ordered questions to ask the witness or to act as a 
reminder as to certain areas, of questioning that must be covered 
during the examination. 

Referring to Fig. 7a, while . of f -line, upon entering the 

25 outline mode, the examining attorney uses the attorney terminal 
15 to create a conventional outline of the names of categories 
and subcategories to be covered at a deposition using an outline 
window 285. As illustrated, an area of law such as "patent law" 
is entered as a hierarchical entry point by assigning it a Roman 

30 numeral as is in typical outlines. Each subcategory of patent 
law such as infringement, invalidity, laches, etc., is then 
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entered thereunder. Similarly, the attorney continues to enter 
sub -subcategories and so on until all of the categories of patent 
law that need to be discovered are in the outline. Next, the 
attorney enters, questions or notes under the appropriate 
5 categories in the outline. For example, under "best mode" the 
attorney may enter the question: "Did you disclose the most 
recent version of your invention in the patent application?" 
After all of the questions (or notes) have been entered, the 
attorney then estimates the amount, of time that should be spent 

10 on each category. This time is entered through a command line 
283 for use by a scheduler (described below in more detail) . 

After entering the entire outline, questions and estimated 
times, the attorney invokes a toggle command on the command line 
283 which causes the outline structure to enter the graphical 

15 tree-structured display mode illustrated in Fig. 7b. In this 
mode, the outline window 285 displays the created outline in a 

* tree-structured format. Each category and subcategory can be 
easily discerned as well as the total estimated time to finish 
each hierarchical level. For example, in a block 287, the area 

20 of law, patent law, can be liound to have an estimated time of 
eight (8:00) hours to complete. This timie is comprised of one 
hour and thirty minutes (1:30) to complete "laches" at a block 
289, five hours (5:00) to complete "invalidity" at a block 291, 
and one hour and thirty minutes (1:30) to complete "willfulness" 

25 at a block 293. In the preferred embodiment , the underlying 
questions or notes associated with each category or sub- category 
are not displayed in the tree-structured display . Toggling 
between the standard outline display and the tree-structured 
display can be repeated until preparation for the deposition is 

30 complete. 
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In addition to generating entire outlines from scratch, 
standard, reusable outlines pertaining to specific areas of law 
may be effectively utilized as a default. Once the outline is 
created, it may be re-used in subsequent depositions by merely 

. 5 adding or deleting questions where applicable. 

Pretyped questions are used during the deposition by both 
the examining and defending attorneys to direct the proceedings . 
In addition, notes can be added detailing the current state of 
the law in each subcategory for reference during a deposition. 

10 Referring back to Fig. 6a, while in the outline mode during 

a deposition, the configuration of windows on screen 251 of the 
examining attorney's terminal 15 is identical to that described 
above with one exception. Instead of displaying Q & A's, the 
window 255 is used for an outline mode display. Particularly, 

15 whenever an outline command is selected from the command line 
259, the window 255 displays the tree-structured outline 
illustrated in Fig. 7b. From this outline, the attorney may 
choose to: 1) toggle to the standard outline, illustrated in Fig. 
7a, to update the outline; 2) evaluate and, if necessary, change 

20 the time estimates based on the deposition progression; or 3) 
select a category of questions and notes for immediate use. 

By double -clicking the mouse button on the desired category 
(or through command selection via the . command line 259), the 
pretyped questions and notes relating to the selected category 

25 are assembled onto a stack and displayed in the communication 
window 257 for immediate use. Upon user setup selection, the 
pretyped communications can be pushed onto the stack or be placed 
into a new independent stack which is solely displayed on the 
window 257. With, the later selection, an individual stack is 

30 constructed for each and every subcategory in the outline. 
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As with off-line preparation, during the deposition, notes 
may be modified or added. Similarly, new categories may be added 
so that existing notes may be placed thereunder. To move an 
existing note, the note need only be selected via the window 257 
5 and transferred by selecting a category via the window 256. . 

In an alternate embodiment, color coding is used to aid the 
identification of areas of remaining inquiry. For example: a 
grey category name indicates that all of the notes thereunder 
have been addressed; a yellow category name indicates that all 

10 of the notes thereunder need to be addressed; and a blue category 
name indicates that some of the 5 notes thereunder need to be 
addressed. Similar color coding can be used for the specific 
notes. Colors can change automatically, upon detecting that all 
notes have been used, or manually by the attorney upon exhausting 

15 an inquiry without using all the questions or notes. 

In addition, the terminals 15 and 17 record the range of 
the Q & A numbers which occur while within each specific category 
or subcategory. The range of Q & A's recorded can thereafter be 
used to aid in the review of areas of inquiry which are scattered 

20 throughout a transcript. 

Objections to form must be seasonable, therefore timing is 
of utmost importance. To speed up the objection process, 
software routines (herein "AI routines") analyze the form of each 
question and the content of the answer during and immediately 

25 after translation to provide various potential objections that 

. the examining attorney may want to take into consideration in 
attempting to achieving good evidentiary form. For example, to 
identify potential mischaracterizations , a real time (i.e., as 
the questions are transcribed) evaluation of each question is 

30 performed to identify phrases such as "you said", "you stated", 
"you tell me", "you told me", "you are telling me", or "you say". 
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Similarly, the use of double negatives can be identified by 
examining a single sentence for multiple occurrences of words 
such as "no", "none" or "not" and , words ending with "n't". 
Potential hearsay may also be indicated by examining each 
5 sentence of an answer for multiple occurrences of "said", 
"stated", "told" or "telling" which are not modified by a 
preceeding "I". 

Through this evaluation or "filtering" process, if an 
indication for an objection is found, the AI routines immediately 

10 send a potential objection to the communication windows 
indicating that the question possibly mischaracterizes the 
witnesses earlier testimony. In response to a potential 
objection communication, the defending attorney may object, and 
may also initiate an immediate search for the mischaracterized 

15 statement by opening the communication and pressing the scan up 
key 279, invoking a search based on the current question of 
previous answers. In this way, mischaracterizations may be 
avoided or corrected. 

The AI routines for detecting potential objections are 

20 preferrably associated with the transcription functionality so 
as to provide the most seasonable potential objection possible, 
i.e., during the transcription process on the CAT system. 
However, the routines may also exist separate from transcription 
functionality, for example at each attorney terminal; If within 

25 the CAT system, the CAT system broadcasts potential objection 
communications to each terminal on each communication link. If 
the AI routines are within each terminal, the routines merely 
display potential objection communications as messages "From: AI" 
in the communication window. 

30 As illustrated in Fig. 8, associate examining attorney 

terminal 17 includes a keyboard 301 having alphabetical keys 
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arranged in a standard typing keyboard key pattern. As with the 
examining attorney's terminal 15, the associate examining 
attorney terminal 17 includes a screen 303 which is split into 
a transcription window 305 and a communication window 307 which 
5 have a common command line 309. While in the deposition mode, 
the terminal 17 displays the alphabetic and numeric text 
representing the spoken words in the transcription window 305, 
and the window 307 provides for input and display of messages, 
notes, etc. In the outline mode, the window 305 performs the 
10 hierarchical outlining functions described more fully above in 
relation to the terminal 15. 



examining attorney as the deposition is being carried out. The 



15: described previously regarding communication in the opposite 
direction with one enhancement . The associate's terminal 17 
provides the capability of viewing, modifying,, and sorting the 
communication currently present on the attorney terminal 15. 
Such capability allows the associate attorney to help organize 

20 the terminal 15,. which has added importance for an examining 
attorney with little computer literacy. Moreover, the associate 
attorney may also take full control of the first chair attorney' s 
terminal 15 by requesting the complete viewing status of the 
first chair's terminal via the communication link. Upon 

25 receiving the viewing status, the second chair associate 
attorney's terminal displays the first chairs status, duplicating 
the first chairs screen. By echoing commands selection between 
the terminals, the associate attorney can control the display of 
the first chair's terminal for the examining attorney. Alternate 

30 schemes of control may also be used with the goal of permitting 



The associate attorney may also send messages to the 



functionality of such communication is identical to that 
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the second chair attorney to manage the display of the first 
chair attorney's terminal via the second chair's terminal. 

AI objections are always associated with specific questions 
or answers . Messages may be associated upon selection via the 
5 command lines so that the recipient can place the question in 
context. Notes can be similarly associated. Important messages, 
notes, or objections can be recorded in a supplemental file by 
using the annotate command from the command line. Annotating 
creates a file of annotations which may or may not be associated 

10 with a specific Q & A, which may be stored with the transcript 
for later retrieval and review. 

Specifically, referring to Fig. 6B, associations with a 
given question and answer is shown preceding each type of 
communication. As illustrated for example, the message from the 

15 second chair, associate attorney, is associated with the fifty 
seventh question and answer (QA#57) . Similarly, the potential 
objection message from the system's AI. filters shows an 
association with QA#42. 

To display a given communication in full, the attorney 

20 merely double-clicks on the communication. This causes the 
conversion of the communication window 257 from a list box to an 
edit window format, then causes the display of the full text of 
the given communication therein. A communication is considered 
"selected" in while in the list box format when it is high- 

25 lighted. By a single click of the mouse or via the arrow keys, 
selective high-lighting can be applied to a given communication. 
A communication is also considered "selected" while it is 
currently being displayed in the full text format. 

To associate a communication such as an outgoing message 

30 (TO: 2nd for example) to a question and answer, the attorney: 1) 
displays the appropriate question and answer in the window 255; 
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2) selects the communication in the window 257; and 3) invokes 
an associate . command via the command line 259. In the 
alternative, the attorney may select the communication and, while 
holding down the left-mouse button, drag an iconized 
5 representation of the communication to the window 257 and release 
the mouse button. This action is interpreted by the terminal 15 
as an invocation of the associate command. Similarly, a question 
and answer can be selected for dragging and dropping on a given 
communication for association. Once associated, the attorney may 

10 select a send command via the command line 259 to cause the 
message and the association to be transmitted along the 
communication link to another attorney, client, expert, etc. 

To display the associated question and answer, the attorney 
first selects a given communication then invokes a display 

15 association command via a right mouse click or through the 
. /..command line 259. Upon invocation, the terminal 15 responds to 
the display association command by causing the display of the 
associated question and answer in the window 255. In this way, 
the attorney may determine the context of the communication. 

20 Thus, upon receiving a message from the associate attorney, the 
f irst, chair attorney may not only display the . message but may 
also automatically display the associated question and answer 
from which the message was derived. The association, which 
places the message in context, supplements the understanding of 

25 such messages permitting messages of a much more abbreviated form 
to be sent. Abbreviated messaging also minimizes the time needed 
for message construction and transmittal. As a result, such 
messaging is more likely to reach the recipient in time for 
optimal use of the message ("seasonable") . 

30 Furthermore, although a visual indication of the 

association (i.e., the mark "QA#57" for example) is only shown 
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in the window 257, a similar- indication could be displayed in 
association with the questions and answers in the window 255. 
Thus, by "selecting" a given question and answer then invoking 
the display association command, the corresponding communication 
5 could be displayed in the communication window 257 . In addition, 
multiple associations might also be made. For example, a 
remotely located client might create a message relating to a 
scanned document (being displayed on the client ( s terminal in 
response to a display document command via the command line 259) 

10 or a note and a question and answer received in real-time. By 
selecting the constituents, invoking the associate command and 
selecting the transmit command, the client can send the message 
and the context to the examining attorney's terminal. Upon 
receipt, the examining attorney may selectively display not only 

15 the text of the message but also the associated context of that 
message, i.e., the associated document or note and question and 
answer. 

A demeanor marker command also provides for annotating a 
specific Q & A. Noticing that the witness turns red or begins 

20 to stutter, fidget, or raise their voice in response to a 
specific question, an attorney can select the demeanor command 
. which presents a pretyped list of demeanor indications such as 
"fidgeting" or those listed which may be immediately associated 
with the current Q & A and stored as an annotation to the 

25 transcript. Similarly, a conferral command can be selected to 
annotate that the witness and opposing attorney conferred off the 
record regarding an associated Q & A. Additionally, although not 
generally useful during the deposition because they may be 
overheard, voice recorded notes or annotations may also be added 

30 by an attorney reviewing the transcript away from the deposition. 
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Such annotations may be selectively displayed or played back via 
the attorney terminal's command line. 

In the communication windows of each terminal, a stack 
containing a series of common pretyped messages ("common message 
5 stack") can be quickly accessed for selective transmittal. For 
example, upon invocation of a preset message command from the 
command line, a series of common messages such as "What was the 
date, time, year?", "Were there any witnesses to the event?", or 
"What is his name, address, employer?" can be pretyped and listed 

10 in a message stack format in the communication window 307 of 
terminal 17. While receiving transcription in real time, the 
associate attorney need only select the desired pretyped message 
from the stack, associate the message with the current question 
and answer (if desired), and select the send command. Thus, the 

15 pretyped message has a much greater chance for seasonably 
reaching the first chair terminal in an understandable form than 
would a non-pretyped message. Similarly, pretyped potential 
objection messages or notes can be added for quick access to the 
common message stack in the communication window of the associate 

20 defending attorney's terminal 18. The common message stack at 
each terminal can be customized by editing, adding or deleting 
messages therefrom. 

A time command is also available for the command windows of 
the attorney terminals. Executing the time command enables the 

25 attorney to identify delay times between the end of a specific 
question and the beginning of the witness's answer. Similarly, 
using the time command, witness and opposing attorney conferral 
time can be recorded. Underlying this command, a system timing 
routine operates to log event timing in a separate data file 

30 which the time command can access for specific requests. 
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The system timing routine also operates as an alarm clock 
for scheduling a deposition. Referring back to Figs. 7a and 7b, 
using the alarm command while preparing a hierarchical outline 
for the deposition, the attorney may allocate a projected amount 
5 , of time for each category or subcategory. The projections can 
be monitored by the system timing routine, and indicate to the 
. attorney through a system communication that projections are not 
being held. Specifically, while under a particular category, the 
system timing routine begins to count down the time allocated to 

10 the subcategory as described above. If the attorney completes 
the category before time runs out, the extra time, will be added 
to a buffer. If instead time runs out, the alarm clock sends a 
communication to the attorney indicating that the projected time 
has expired and indicates the buffer status. The buffer is 

15 periodically updated by subtracting the time which lapses beyond 
the projected value. If after receiving the system communication 
the attorney remains in a category past the projected time 
period, the alarm clock sends a new system communication every 
thirty (30) minutes (as a default value) . 

2 0 As the deposition progresses, whenever the attorney enters 

the tree-structured outline mode, the time value displayed for 
each category, as illustrated in Fig. 7b, reflects the time 
remaining in each specific category. The time value for the 
category currently selected (and its parents in the tree- 

25 structure) continually updates so as to display the. scheduler's 
minute by minute count down. Rescheduling or other time value 
changes are automatically reflected throughout the entire tree- 
. structured display, enabling better time management of the 
deposition. 

30 When a confidential restriction is placed on specific 

subject matter of a deposition, the examining attorney often 
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forgets to remove the restriction when the subject has changed. 
By selecting the alarm clock command and then the subsequently- 
listed confidentiality wake-up command, the terminal 15 
automatically sends a system communication every fifteen minutes 
5 (can be changed through a set up screen) to remind the examining 
attorney to clear the restriction from the transcript if 
possible; In addition, upon selecting a. different category or 
subcategory in the hierarchical outline, a system communication 
is also immediately transmitted as a reminder. 'The 

10 confidentiality command also provides a mechanism for disabling 
and enabling local network communication from reaching off-site 
client terminals, unless specifically directed thereto. 

In one embodiment of the present invention, translated 
sentences are communicated from CAT system 13 via serial 

15 communication along communication link 21 to terminal 15 and 
communication link 23 to terminal 17. Communications between 
terminals 15 and 17 utilize the two independent communication 
links 21 and 23. In another embodiment, the communication links 
21 and 23 are interconnected and function as a single 

20 communication link (bus) using a TCP- IP protocol on an ethernet 
. coaxial cable link using an ethernet board in each terminal. Two 
ethernet boards are desirable in the reporter terminal so that 
isolation can be maintained between the opposing sides. A 
software package "Berkeley Sockets", developed at the University 

25 of California, Berkeley, is the preferred software base for 
maintaining the communication flow along the link. Packages 
implementing "Berkeley Sockets" are available through various 
third party vendors, including for example FTP Systems, Inc., 
Burlington, Mass, The communication links 20 and 22 are 

30 interconnected and function in a similar manner via the second 
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ethernet board .in the reporter terminal. Other hard- wired or 
wireless communication may also be implemented. 

Two issues of isolation must be addressed by the 
communication links in the present invention. First, the court 
5 reporter stores all of the keystrokes pressed and all of the 
transcribed alphabetic and numeric text in the CAT system 13 . 
These files must not be modifiable by any terminal on any 
down-line communication link. Such modifications would corrupt 
these f iles . and the transcript would not be certifiable. 

10 Therefore, the communication out of CAT system 13 should only be 
that necessary to transmit copies of the stored files. More 
importantly, communication into CAT system 13 should only be that 
necessary to request transmission. Although additional 
communication . features into and out of CAT system 13 may be 

15 added, they are not preferred. 

Second, isolation must be established between the defending 
and examining terminals. Under no circumstances can the messages 
between the examining terminals be intercepted by. the defending 
terminals. This requires that the communication links must be 

20 isolated. Isolation may be accomplished by using an independent 
hard-wired communication link as is. illustrated in Fig. 1 via a 
serial or coaxial link, for example. Encryption of the 
communication exchanged may be used as a safeguard. However, if 
a wireless link is used, encryption becomes a requirement. 

25 Although various encryption schemes are contemplated, a preferred 
scheme involves the use of direct-sequence spread spectrum 
transmission. Such transceivers are relatively inexpensive and, 
by their very nature, provide encryption. Other exemplary 
communication links which might be implemented include token 

30 ring, parallel, infrared, RF (radio frequency), etc. 
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Fig. 9 illustrates additional featurea of: t>he present 
invention which the court reporter and attorneys may utilize. 
As stated previously, the court reporter usually tapes the entire 
deposition or trial as a back up to the stenographic recording, 
5 and uses the audio tape to help in translating the proceedings. 
To avoid clumsy and time consuming searching through the audio 
tape to locate Q & A's, a synchronization scheme has been added 
to electronically associate each stenographically recorded Q & 
A with the actual position on the audio tape of the audio of the 

10 Q & A. With synchronization, the court reporter can quickly and 
easily locate audio on the tape. 

Specifically, a tape recorder 351 is attached via a 
communication link. 353 to the reporter terminal 13. Upon 
receiving an entire Q & A, the reporter terminal 13 requests and 

15 receives a position indication from, the tape recorder 351. The 
marker position is generated by the tape . recorder by a tape 
length counter 355 and associated circuitry. The reporter 
terminal 13 stores and associates each such position indication 
with the corresponding keystrokes. Thereafter, when necessary, 

20 the court reporter can display the position indicator and. quickly 
locate the desired audio manually. Alternatively, communication 
link 353 can be used by reporter terminal 13 to communicate 
control signals to the tape recorder 351 so as to automatically 
position and play-back previously recorded audio upon request. 

25 Instead of receiving position indications from the tape 

recorder 351, the reporter terminal 13 may send an audio signal 
representing an upcoming Q & A number to recorder 351 for 
recording. This audio signal may be either a voice-synthesized 
indication of the number count, aiding a manual search, or a 

3 0 digital data stream that is detectable by the recorder 351 using 
low or high speed searching directed via terminal 13. 
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Similarly, a video camera 361 may also be used to video 
tape of proceedings. Although video camera 361 may be able to 
independently store and play audio tapes, a separate video 
recorder, video recorder 363, is illustrated as being connected 
5 via a communication link 365 for this purpose. As with the audio 
recorder, video recorder 367 can be used in a synchronization 
scheme for associating and acciessing video events with the 
transcribed text through communication link 369. A television 
monitor 371 provides for playing back recorded video information. 

10 If the video or audio system is fully automated through the 

control of reporter terminal 13, upon a request from a judge or 
attorney to read back a Q & A, the court reporter may play back 
the associated audio and video by merely locating the requested 
Q & A on reporter terminal 13, possibly through a lexical search, 

15 and selecting a play-back command. 

Also contemplated is the use object-oriented association of 
recordings using video frame grabbers and sound grabbers resident 
in reporter terminal 13 which digitally sample, store, associate 
and synchronize the video and audio of a proceeding. 

20 Synchronization would take place using a direct association of 
the transcribed text in a common programming object with the 
audio and video frames. 

Using position synchronization for locating and playing 
audio and video also proves to be a valuable tool for the 

25 attorney. If the attorney desires to play portions of the audio 
or the video, for example, during trial, the attorney may use the 
lexical searching capabilities of the attorney terminals to 
locate the desired Q & A's, and may then automatically play the 
associated audio or video back for the judge and jury. 

30 Referring to Fig. 10, a court room scenario is shown using 

the present invention. A first chair, examining attorney 4 01 
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holds a portable computer tablet 403 which has a wireless 
transceiver 405 attached thereto. The tablet 403 may provide 
pen-based or touch sensitive input and acts as a direct 
substitute to the attorney's paper note pad. In one embodiment, 
5 the wireless communication link is established using RF (radio 
frequency) , spread- spectrum transmissions for the inherent data 
encryption associated therewith. As stated previously, other 
types of RF, ultrasonic or infrared transmission and data 
encryption schemes might also be used. 

10 To guarantee isolation of . the court reporter's 

transcription files, the terminal 13 only manages outgoing 
communication and requests therefore as described previously in 
relation to the configuration of Fig. 1. As explained above, 
although such communication exchanges involving the reporter 

15 terminal 13 do not require encryption, secure communications 

— between the attorney terminals must exist to prevent interception 
by the opposing side (not shown). As illustrated, secure 
communication is accomplished between the terminals 15 and 17 
using the independent hard-wired communication link 23. 

20 Encryption using a pseudo-random, secret spreading code common 
only to transceivers 405 and 409 implementing direct-sequence 
spread spectrum transmission secures the communication link 
between the attorney terminal 403 and either of the terminals 15 
and 17. Other encryption schemes may also be used. 

25 Wireless transmission may also be used by additional 

wireless transceivers (not shown) for use by additional on-site 
attorneys staffing the lawsuit. A communication line 411 may be 
used to establish a communication link with additional terminals 
used by attorneys, client's, or experts who are not in the court 

30 room, i.e. are remotely located off-site. Although the 
communication line 411 is a telephone line attaching to an 
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internal modem of terminal. 17, other communication methods such 
as a cellular or other. RF link might be used. The communication 
line 411 provides for the active . participation of off -site 
persons who might otherwise be unable, because of cost or time, 
5 to attend. 

The local attorney terminals 403,. 15, and 17 can control 
the flow of data through communication line 411. For example, 
if confidential information is being described, the attorney 
terminal 15 can block transmission off-site to a client by 

10 disabling outgoing communication via the off-site link by 
invoking a confidential command on the command line 259. 
Alternately, the CAT terminal 13 can place a confidential 
information restriction on transcribed questions and answers, and 
prevent : transcribed text transmission from the court or 

15 deposition room* Instead of sending the transcribed text, 
replacement text is transmitted indicating the current question 
and answer number and the status. The receiving off -site 
terminals display each question and answer number with 
"Confidential" status message instead of the actual transcribed 

20 text . 

Together, the transceivers 407, 405, and 409 and the links 
23 and 411 form a single communication link between the 
terminals. Channel isolation exists between the examining 
attorneys. File isolation exists at the CAT terminal 13. The 

25 characteristics of this single, overall communication link is 
transparent to the attorneys operating the terminals, they need 
only choose a destination and can send messages. Although not 
shown, the opposing attorneys also possess a similar 
communication system on a second, single communication link. 

30 More specifically, the first chair, attorney 401, uses 

portable tablet 403 to record notes, receive translated Q & A's 
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and exchange messages with the second chair at terminal 15 . The 
first chair can either direct the message, to one of the 
supporting attorneys at a terminal or globally send messages to 
all available attorneys at all of the terminals on the link. 
5 Similarly, messages can be received, from such terminals. 
Although not necessary, a communication pathway hierarchy is 
preferred wherein all messages directed to the first chair from 
lower numbered chairs are intercepted by the second chair 
terminal for screening and redirection.. In this way, the second 

10 chair attorney prevents the first chair attorney from being 
distracted by unnecessary, unimportant, or misdirected messages . 

Similarly, if desired, interception at the second chair 
terminal can also occur flowing from the first chair to the lower 
numbered chairs. By doing so, the first chair attorney need not 

15 be bothered with specific task allocation created by a given 

request. The first chair attorney need only send the message to 

-the second chair, or to whomever the attorney believes the task 
should be performed by, and the second chair attorney terminal 
intercepts and provides for rerouting if necessary. Additional 

20 levels of hierarchy between the second and lower numbered chairs 
may also be implemented. 

A terminal such* as portable tablet 403 or attorney terminal 
15 might also be used by the judge during and after trial for 
reviewing and analyzing the lawsuit. When facing a ruling on the 

25 matter, the judge might also use the features described above to 
determine whether proper evidentiary form was achieved in the Q 
&. A. Similarly, for review and analysis, the jury might be 
provided such terminals during trial or deliberation to help 
reach their decisions. Especially beneficial in either situation 

30 would be the ability for audio and/or video review as described 
in relation to Fig. 9 above. 
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Fig. 11 illustrates an alternate embodiment wherein, 
instead of responding indirectly to stenographic keystroke 
translation, the reporter terminal 13 directly performs voice 
transcription and association. Added to the reporter terminal 
5 13, a microphone 451 generates an. analog voice signal 
representative of the words spoken during a proceeding. The 
analog voice signal passes along a communication link 453 to an 
A/D (analog to digital) converter board housed within reporter 
terminal -13". This A/D converter samples and digitally converts 

10 the analog voice signal to a series of digital values 
representative of the words spoken at the proceeding. Linear 
predictive coding ("LPC") techniques are then used either 
remotely (through an off-site computer link) or locally (within 
reporter terminal 13) to transcribe the deposition. 

15 ; Additionally, the reporter terminal 13 may also use the 
stenographic keystroke input as a basis for training the voice 
transcription system. Specifically, the reporter terminal 13 
breaks the digital voice signal into segments, and, using LPC 
techniques, synchronizes through association groups of the 

2 0 segments with corresponding keystrokes from the stenographic 
recorder 11. The process of associating the segmented digital 
voice signal with the corresponding stenographic keystrokes 
provides a method for training the voice transcription of the 
reporter terminal 13. Once trained, voice transcription may more 

25 effectively supersede the stenographic recorder 11 in generating 
a written transcript of the proceeding. 

Fig. 12 is used to illustrate two alternate embodiments 
where the court reporter is not linked to the transcription 
network. In a first embodiment, voice transcription is directly 

30 managed by an attorney terminal 555 using a microphone 551 via 
a line 559. Voice transcription can occur either locally, within 
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terminal 555, or remotely via a communication link 565. 
Additionally, as described previously, transcription, 
association, synchronization, and message exchanges would occur 
between terminal 555 and an associates terminal 563. 
5 A second embodiment is illustrated wherein terminals 555 

and 563 provide for message exchange and message association with 
the corresponding recorded audio on the tape, transcription not 
being available. Message association would occur using by either 
retrieving and storing the position indication from recorder 553, 
10 or sending for recording, audio or digital signals for manual of 
automated play back. A more detailed description of the 
functionality of such message association and play-back can be 
found above . 

Fig. 13 is a perspective view of another embodiment of the 
15 present invention. Instead of using a keyboard, the attorney 

- terminals 15 and 17 might adopt pen-based or touch-sensitive 

' screen input. Pens 601 and 603 and the screens 251 and 303 
provide an attorney a substitute for a pad and Post -it® brand 
notes. The ' associate attorney may open a computerized 
20 representation of the Post-it® brand note (a "computer note"), 
scribble a message thereon using the pen 603, and press the send 
command on the command line 309. Unlike actual Post-it® brand 
notes, computer notes can be expanded. Each expanded computer 
note is displayed in the default size yet has an associated 
25 scroll-bar and expand and reduce buttons that can be manipulated 
so that an oversized message can be fully read. 

The terminal 17 directly sends the entire bit -mapped image 
of the computer note to the attorney terminal 15 for display in 
a tiled format (cascading may also be selected as a default) on 
30 the screen 251. The attorney operating the terminal 15 may then 
modify, delete, move, etc., the received computer note using the 

SUBSTITUTE SHEET (RULE 26) Ei006016 



WO 94/22246 PCT/US94/03218 

-62- 

pen 601 and. the commands from the command line 259. Message 
exchanges in the opposite direction and note taking, as 
previously, described, works in. a similar manner. Moreover, 
instead of sending a bit -map of the computer note, handwriting 
5 recognition software may be used to convert the note to a text 
format to make the notes more legible to the recipient. The 
recognition software may be selected directly through command 
line 259 or through a set-up default. 

The transcription network of Fig. 13 is illustrated to 

10 operate without transcription input; however,, if present, 
transcribed text can also be displayed by opening transcription 
windows as described above. 

As. with the previous embodiments, the associates terminal 
17 has the ability to .display and manipulate the messages 

15 displayed on the attorney terminal 15. For example, the 
associate can: 1) delete computer notes that have been used or 
become stale; 2) replace incomplete or unrecognizable notes; 3) 
rearrange the tiled or cascaded notes to better communicate 
priority or to organize subject matter; or 4) modify the 

20 significance indicator causing message border colors to change 
or flash (described more fully above) on the attorney terminal 
15 indicating message priority. 

Fig. 14 illustrates a flow-chart describing the 
communication protocol used by the attorney terminals for 

25 . communicating with the reporter terminal (or CAT system) 21. 
Ordinarily, at the completion of a transcription of a full Q&A, 
the reporter terminal 13 transmits the Q&A down- line to along 
communication links 21 and 20, for example. If the attorney 
terminals, such as the terminals 15, 17, 16 and 18, are 

30 listening, they receive and update their screen displays 
appropriately. A specific protocol has be implemented to 
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accommodate : 1) requests by the attorneys for incomplete, 
preliminary translations; 2) attorney terminals which have not 
"heard" all of the Q&A transmissions; and 3) screening of Q&A's 
by a senior chaired attorney to those down- line such as remote, 
5 client terminals. 

Specifically, as represented by a begin block 651, upon 
connecting to the communication link with the reporter terminal, 
the attorney terminals initiate the communication protocol. At 
a block 653, the attorney terminals construct a log-in packet, 

10 a packet of bytes of digital information, which consists of: 1) 
a source identifier, identifying itself as the source of the 
communication; 2) a destination identifier, identifying the 
reporter terminal as the destination; and 3) a pointer indicating 
the attorney terminal's current position, i.e., the pointer 

15 identifies the last Q&A that the attorney terminal has received 
from the reporter terminal . 

The pointer to the last or most recent Q&A is referred to 
hereafter as a "self . current " pointer. If an attorney terminal 
is logging in for the first time in a deposition, the value of 

20 the "self .current" pointer is equal to zero (0) to indicate that 
no Q&A's have yet been received. Likewise, if- the attorney 
terminal is logging in a second or subsequent time (because of 
either an accidental or purposeful disconnection) , the 
"self . current" pointer is set to the last Q&A number received 

25 prior to the disconnection. 

As represented by a block 655, the attorney terminal sends 
the log- in packet to the reporter terminal, and waits for a 
response. Upon receiving the log- in packet, the reporter 
terminal enters the source identifier into a log-in list, stores 

30 the "self . current " pointer, and transmits a responding packet 
containing a source identifier, destination identifier, and the 
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number of the most recently translated Q&A. The most recent Q&A 
number is maintained by the reporter terminal's "rptr .current" 
pointer. 

Upon' receipt of the response packet, as illustrated by a 
5 block 657, the attorney terminal first determines the number of 
outstanding packets needed. This is done by subtracting the 
attorney terminal's "self . current " pointer value from the 
received, reporter terminal' s "rptr . current " pointer value . The 
resulting difference, (the number of needed Q&A's) is then 

10 compared to a threshold value. If the difference is less than 
the threshold value (a "yes" to the decision at block 657), the 
attorney terminal builds an update request packet, containing 
source, destination and update indicators, and transmits the 
packet as represented by blocks 661 and 663, respectively. In 

15 this instance, the update request packet is very similar to the 
log- in packet, containing: 1) a source identifier; 2) destination 
identifier; and 3) pointer to the attorney terminal's current 
position. 

If the difference. is greater than the threshold value (a 
20 "no" at block 657) , the attorney terminal prompts the attorney 
using the attorney terminal for the range of Q&A's that are 
actually necessary, as represented by a block 659. The range is 
selected by choosing the earliest Q&A desired. The range would 
then be from the earliest Q&A to the most current Q&A as 
25 determined by the reporter terminal. The prompting for a range 
enables the attorney to evaluate the number of outstanding Q&A' s 
that have not been received, and request only those that are 
truly needed. The thresholding prevents large deposition file 
downloading . to late -coming attorneys where unnecessary. 
30 Thereafter, at the block 661, the attorney terminal constructs 
an update request packet using the selected range, and transmits 
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it to the reporter terminal at the block 663 .. In this instance, 
the update request>packet contains: 1) a source identifier; 2) 
destination identifier; and 3) the number of the earliest Q&A 
desired. 

5 At block 663, after transmitting the update request , the 

.attorney terminal, awaits an update. The reporter terminal 
responds by sending the requested range. However, if the 
requesting attorney terminal or a higher chaired attorney has 
placed aQ, A or Q&A restriction upon the requesting terminal for 

10 all or part of the requested range of Q&A's, the reporter 
terminal transmits substituted text, such as "Restricted 
Viewing!" instead of the actual text translation where necessary. 
The reporter terminal tracks restrictions in the log-in table. 
Similarly, whenever a Q&A translation is complete, the reporter 

15 terminal only transmits the text of the translation to those 
-attorney terminals having no restrictions placed thereon. To 
■ those attorney terminals with restrictions placed thereon, the 
substitute text is transmitted. 

Upon receiving the requested Q&A' s or substitute text 

20 therefor, as represented by a block 665, the attorney terminal 
stores the requested Q&A' s and updates its "self . current " 
pointer. At this point, the attorney terminal is considered up 
to date. Thereafter, both periodically and upon an attorney's 
direct request (in response to specific keyboard or mouse input) , 

25 the attorney terminal constructs an update packet, at a block 
667, and repeats the cycle beginning with block 655. This 
looping cycle ensures that the attorney terminal stays up to 
date, even if the attorney terminal for any reason temporarily 
stops monitoring the communication link during the deposition. 

30 Whenever a request packet is received which contains a 

"self . current " pointer which is equal to the reporter terminal's 
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n rptr. current " pointer, the current Q&A pointed to by the 
"rptr . current " pointer is resent along with any partial 
translation for the next Q&A that has been completed. The 
current Q&A is always resent to make sure that, the attorney 
5 terminal possesses the full translation thereof (in case only a 
partial translation is currently stored) . The transmission of 
the partial translation of the subsequent Q&A allows the 
specifically requesting attorney to display the ongoing 
translation. 

10 The value of the threshold depends on nature of the 

communication network and user preference. If the network is 
sluggish, the threshold should be set lower to off-load competing 
communication on the channel. To accommodate each user's 
preference, control of the threshold value is be placed at each 

15 attorney terminal. 

In addition, although the thresholding decision is made by 
the attorney terminal as. represented by the block 657, in an 
alternate embodiment, the reporter terminal makes the decision. 
Specifically, referring. to Fig. 15, a flow-chart of an alternate 

20 embodiment of the software protocol used by the reporter terminal 
13 is shown. Upon receipt of either a log-in or update request 
packet, the reporter terminal initiates its response as 
represented by a begin block 681. At a block 683,. the source 
indicator is compared to the log-in list to determine whether the 

25 currently requesting attorney terminal has been logged therein. 
If not, the source indicator (or source identifier) is added to 
the log- in list. 

At a decision block 685, the reporter terminal compares the 
. received "self . current" pointer with its "rptr . current " pointer. 

30 If the difference between the two is less than a threshold value 
(as indicated by the "yes" path) the reporter terminal: 1) 
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checks for viewing restrictions in the requested range, at a 
block 691; 2) constructs a response which includes all of the 
requested Q&A's. with substituted text where restricted, at a 
block 693; 3) transmits the response to. the requesting attorney 
5 terminal, at the block 695;. -viid 4) exits ;;he rouuine at the end 
block 697. However, if the difference at the decision block 685 
is greater than- the threshold value (indicated by the "no" 
pathway) , prior to encountering blocks 691, 693, 695 and 697, the 
reporter terminal first prepares and sends a range packet inquiry 

10 to the requesting terminal and awaits an update response 
therefrom, respectively at blocks 687 and 689. The range inquiry 
packet indicates the total range of outstanding Q&A's. The 
attorney terminal responds by prompting the user for the earliest 
desired Q&A, constructs another update request packet, and 

15 transmits it to the reporter terminal which awaits its receipt 
— at the block 689. From this responsive update request packet, 
• the reporter terminal completes the protocol through the 
remaining blocks as described above. 

Additionally, it is obvious that the embodiments of the 

20 present invention described hereinabove are merely illustrative 
and that other modifications and adaptations may be made without 
departing from the scope of the appended claims. 
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1. A transcription network comprising: 

transcription means for generating transcript signals 
representative of spoken words; 

an attorney's terminal having a display screen; 
5 an associate's terminal having a display screen; 

a communication link between said terminals; 

each terminal receives the transcript signals generated by 
said transcription means and displays the transcript signals in 
real time; 

10 said associate's terminal providing for the selective 

association of messages with transcript signals being displayed 
in real time for subsequent transmission to said attorney's 
terminal; and 

said attorney' s terminal receiving said associated messages 
15 via the communication link from said associate's terminal, and, 
through access to said associated messages, providing for the 
display of the associated transcript signals. 

2. A transcription network comprising: 

transcription means for generating transcript signals 
representative of spoken words; 

an attorney's terminal having a display screen; 
5 an associate's terminal having a display screen; 

a set of pretyped messages accessable from said associates 
terminal ; 

a communication link between said terminals; 
each terminal receives the transcript signals generated by 
10 said transcription means and displays the transcript signals in 
real time; 
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said associate's terminal providing for the selection of 
ones of said set of pretyped messages relating to the transcript 
signals being displayed in real time for subsequent transmission 
to said attorney's terminal. 

3. A transcription system comprising: 

transcription means for producing textual representations 
of spoken words in real time; 

a screen which displays the textual representations of 
5 spoken words produced by said transcription means; and 

means for performing a real time evaluation of the textual 
> representations of the spoken words and selectively responding 
to the evaluation by causing the display of objections to the 
textual representations on said screen. 

.-.4. A transcription network comprising:. 
■ a communication link; 

a first chair attorney terminal capable of receiving 
communications via said communication link; 
5 an associate terminal capable of communication via said 

communication link with said first chair Attorney terminal; 

an additional terminal capable of communication via said 
communication link; 

means for intercepting communications originating from said 
10 additional terminal destined for said first chair terminal for 
review and selective relaying via said associate terminal. 

5. A transcription network comprising: 

transcription means for producing transcript signals 
representative of spoken words in real time; 
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a first chair attorney terminal for displaying transcript 
signals produced by said transcription means; 

. a remote terminal capable. of displaying transcript signals 
produced by said transcription means; 
5 means for selectively preventing the display on said remote 

.terminal of transcript signals deemed confidential. 



6. A transcription network comprising: 

transcription means for generating transcript signals 
representative of spoken words in real time; 

an attorney's terminal having a display; 
5 an associate's terminal having a display; 

a communication link between said terminals; 

each terminal receives and displays transcript signals 
generated by said transcription means; 

said associate's terminal capable of generating and sending 
10 messages for display on said attorney' s terminal; and 

means associated with said, associate's terminal for 
controlling the display of messages on the attorney's terminal 
from the associate's terminal. 

7. A transcription network for examining a witness comprising: 
transcription means for generating, transcript signals 

representative of a witness's spoken . words in real time; 

an attorney's terminal positioned in viewing range of the 
5 witness and receiving the transcript signals generated by said 
transcription means and displaying the transcript signals ; 

a set of pretyped demeanor notes accessable via said 
attorney's terminal; and 
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said attorney' s terminal providing for the selective 
association of ones, of said set of pretyped demeanor notes 
relating to the transcript signals being displayed in real time. 

8. A transcription network comprising: 

transcription means for producing transcript signals 
representative of spoken words in real time; 

a plurality of examining terminals receiving and displaying 
5 transcript signals produced by said transcription means; 

a first communication link between said examining 
terminals ; 

a plurality of defending terminals receiving and displaying 
transcript signals produced by said transcription means; 
* 10 a second communication link between said defending 

terminals ; and 

means for isolating said first and second communication 
links. 

9. A transcription network comprising: 

transcription means for generating transcript signals 
representative of spoken words in real time; 

an attorney's terminal receiving the transcript signals 
5 generated by said transcription means for display; 

an associate's terminal receiving the transcript signals 
generated by said transcription means for display; 

said associate's terminal providing for the generation and 
selective association of messages with transcript signals for 
10 transmission to said attorney's terminal; and 

said attorney's terminal providing for the display of 
transcript signals through access to the associated messages 
received from said associate's terminal. 

Ei006026 

SUBSTITUTE SHEET (RULE 26) 




Ei006027 



WO 94/22246 PCT/US94/03218 



2/18 



PAPER 



TAPE: 



35 



41 



PRINT 
ACTUATOR 



33 



COMMUNICATION 
PORT 



39 




KEYBOARD 
SCANNER 





43 



"45 



11 



31 



□ □ □ 



FIG. 2 

Ei006028 



WO 94/2224« 



3/18 



PCT/US94/03218 




Ei006029 



WO 94/22246 



PC17US94/03218 



4/18 



155, 
157, 
159 
161 n 
163 



KEY-STROKE CODE LISTING (KCL) RECORD 



151 



LISTED_KEY-STROKE_CODE 



LISTING_OPTIMIZING_COUNTER 



POINTER TO NEXT *KCL_RECORD ON THIS LEVEL 



POINTER TO *CT_RECORD 



POINTER TO *KCL_RECORD ON NEXT LEVEL 



165 



167 



169 



CORRESPONDING TEXT (CT) RECORD 



CT_STRING 



CT_TEXT_OPTIMIZING_COUNTER 



POINTER TO NEXT *CT_RECORD (HOMONYMS) 



153 



FIG. 4a 



Ei006030 



WO 94/22246 



PCT/US94/03218 



5/18 



KCL RECORD 



KCL RECORD 



200 



CT RECORD 



201 



205 



207- 



CT RECORD 



f 



KCL RECORD KCL RECORD 



202 



v" 203 



KCL RECORD 



CT RECORD 



209 



204 









— > 



FIG. 4b 



Ei006031 



WO 94/22246 



PCT/US94/03218 



223 



6/18 



PHONEME LIBRARY 



220 



221 



Usted_Key-Stroke_Code Corresponding_Phoneme 



225 



- ( W A Z) 


■WAS" 











227 



J 



FIG. 5a 



230 



233 



FOREIGN LANGUAGE TABLE ^~ 231 
ldentifled_CT_String CorrespondlngJTranslation 



235 



— 'good" 


"bueno" 











237 



] 



FIG. 5b 



Ei006032 




Ei006033 



WO 94/22246 



PCT/US94/03218 



8/18 




El006034 



WO 94/22246 



9/18 



PCT/US94/03218 




Ei006035 




Ei006036 



WO 94/22246 



11/18 



PCT/US94/03218 




Ei006037 



WO 94/22246 



PCT/US94/03218 




Ei006038 



WO 94/22246 



PCT/US94/03218 



13/18 




Ei006039 



WO 94/22246 



14/18 



PCT/US94/03218 




Ei006040 



WO 94/22246 



15/18 



PCT/US94/03218 




Ei006041 



WO 94/22246 



16/18 



PCTAJS94/03218 




Ei006042 



WO 94/22246 



PCT7US94/03218 



17/18 




IS 

rptr.current 
- self.current 
< threshold 
? 



no 



653 



build login 
packet using 
self.current 








send p 
and a 
respc 


>acket 
wait a 
>nse 



ask user for 
Q&A update 
range 



655 



657 



yes 



659 



periodically or 
upon request, 
build update 
packet 



661 



build update 
request 
packet 






transmit 
request & 
await update 






store 
& up 
self.c 


data 
►date 
urrent 



663 



665 



667 



Fig. 14 



Ei006043 



WO 94/22246 



PCT/US94/03218 




update login 
list if needed 



is 

rptr.current 
■ self.current 
< threshold 
? 



no 



build & send 
range inquiry 
packet 



await update 
request 
packet 



685 



yes 



687 



689 



check for 
viewing 
restrictions 






construct a 
response 






transr 
respc 


nit the 
Dnse 




691 



693 



695 



697 



Fig. 15 



Ei006044 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US94/032I8 



A CLASSIFICATION OF SUBJECT MATTER 

IPC{5) . :H04K U02 
US CL :380/9; 370/17,110.1 
According to International Patent Classification (IPC) or to both national, classification and IPC 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols) 
U.S. : 380/9; 370/17,110.1 



Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



A 
A 
A 
A 
A 
A 
A 



US, A, 3,557,927 {WRIGHT, ET AL) 26 JANUARY 1971 
US, A, 3,597,538 (BINENBAUM) 03 AUGUST 1971 
US, A, 3,892,915 (BUDWORTH, ET AL) 01 JULY 1975 
US, A, 4,205,351 (MICHALS) 27 MAY 1980 
US, A, 4,692,042 (CUFF, ET AL) 08 SEPTEMBER 1987 
US, A, 4,724,285 (LEFLER, ET AL) 09 FEBRUARY 1988 
US, A, 4,765,764 (LEFLER) 23 AUGUST 1988 



1-9 
1-9 
1-9 
1-9 
1-9 
1-9 
1-9 



fx] Further documents are listed in the continuation of Box C. Q See patent family annex. 



*i:* 
•t.* 



Special categories of cited documents: 

document defining the general Mate of the art which t* not enrwidcrai 
lo be port of particular relevance 

/ 

earlier document published on or after the inlemobonal filing dote 

document which "may throw doubt* on priority chiimjs) of which is 
cited to establish the publication dale of another citation or other 
special rea»on (a* specified) 

document referring to an oral disclosure. u*c. exhibition or other 
means 

document published prior to the mtcmauonal filing date but later than 
the priority date claimed 



bter document published after the international filing dote or priority 
date and not in conflict with the oppfccalion hut cited to understand the 
principle or theory underlying the invention 

document of purlieu hi r relevance; the claimed invention cannot be 
considered novel or cannot he considered to involve an inventive step 
when the document is taken alone 

document of portkulur relevance; the claimed invention cannot be 
considered to involve on inventive step when the document is 
combined with ore or more other such documents, such combination 
being obvious to o person skilled in the art 

document member of the same potent family 



Date of the actual completion of the international search 



29 AUGUST 1994 



Date of mailing of the international search report 

1 4 SEP 1994 



P 1994 



Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington, D.C. 20231 
Facsimile No. (703 ) 308-0463 



Authorized officer 



Form PCT/ISA/210 (second sheel)(July 1992)* 



DAVID CAIN 
Telephone No. (703 ) 308-0463 



Ei006045 



INTERNATIONAL SEARCH REPORT 



International application No. 
PCT/US94/03218 



t .c-.Mitinualion). DOCUMENTS CONSIDERED TO BE RELEVANT 



( '.Memory* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No 



A 
A 

A.P 

A 
A 
A 



US, E, RE.33,337 (LEFLER, ET AL.) 18 SEPTEMBER 1990 
US, A, 5,164,985 (NYSEN, ET AL.) 17 NOVEMBER 1992 
US, A, 5,247,575 (SPRAGUE, ET AL.) 21 SEPTEMBER 1993 
US, A, 5,127,003 (DOLL, ET AL.) 30 JUNE 1992 
US, A, 5,101,402 (CHIU, ET AL.) 31 MARCH 1992 
US, A, 4,041,467 (COTA, ET AL.) 09 AUGUST 1977 



1-9 
1-9 
1-9 
1-9 
1-9 
1-9 



Form PCT/ISA/210 (continuation of second sheet)(July 1992)* 



E«006046 



